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1.0  EXECUTIVE  SUMMARY 


The  Spatial  Light  Rebroadcaster  (SLR)  Architecture  Study  Program  was 
funded  by  Wright  Laboratory  WL/ELOT.  The  performance  period  of  the  program 
was  from  April  19,  1990  to  August  14,  1992.  The  goal  of  the  program  is  to  develop, 
analyze  and  demonstrate  optical  processing  architectures  based  on  SLR  devices. 

1.1  Motivation 

Spatial  Light  Rebroadcasters  may  offer  higher  data  throughput  rales  if  they  can 
be  incorporated  into  avionic’s  systems  to  alleviate  the  bottlenecks  associated  with 
sensor  fusion,  target  classification,  voice  recognition,  and  interprocessor 
communications.  Target  recognition  is  chosen  here  as  generally  representative  of  all 
these  problems  because  they  are  all  plagued  by  a  lack  of  high  speed  "smart" 
interconnection. 

Reconnaissance  and  targeting  are  increasingly  performed  via  electronic 
sensors.  With  the  development  of  smart  weapons  and  Unmanned  Aerial  Vehicles 
(UAV)  as  reconnaissance  and  autonomous  weapons  delivery  platforms,  there  is  a  need 
for  automatic  target  classifiers  (ATC)  that  are  compact  and  consume  little  electrical 
power.  The  computation  and  memory  requirements  of  ATC  algorithms  generally 
increase  with  their  effectiveness.  Moreover,  with  the  emergence  of  critical  mobile 
targets,  the  processing  hardware  must  be  capable  of  very  high  throughput  in  order  to 
meet  the  search  rate  requirement.  With  conventional  electronic  processors,  higher 
computation  speed  can  only  be  achieved  at  the  expense  of  higher  power  consumption. 
At  this  time,  the  Air  Force  does  not  have  any  fieldable  real  time  ATC  for  weapon 
delivery  and  reconnaissance  systems.  Optical  processing  technology  offers  a  potential 
means  to  meet  the  computation  rate  and  the  power  consumption  requirements  of 
current  and  foreseeable  ATC  systems. 
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Most  optical  processing  ^sterns  being  proposed  are  based  on  coherent  optical 
processing  technology  which  requires  a  spatial  light  modulator  (SLM).  Substantial 
efforts  have  been  expended  in  the  development  of  SLMs  to  meet  the  ATC  processing 
requireme'’t  but  the  performance  achieved  thus  far  is  still  seriously  lacking.  A  new 
class  of  optical  devices,  spatial  light  rebroadcasters  (SLR),  have  been  developed 
recently  with  potential  performance  far  exceeding  current  SLMs.  Instead  of  spatially 
modulating  the  input  light  Held,  an  SLR  absorbs  the  incident  radiation  and  re-emits 
when  triggered  to  do  so.  The  triggering  can  be  accomplished  with  an  optical  or 
electrical  signal  that  carries  additional  spatial  information.  The  relationship  between 
the  intensities  of  the  incident  (input),  triggering  (readout)  and  emitted  (output) 
radiations  can  be  used  to  perform  massively  parallel  mathematical  operations. 

In  addition  to  the  incoherent  nature  of  the  radiation  and  the  temporal 
properties,  the  storage  and  transfer  characteristics  of  SLRs  are  also  substantially 
different  from  those  of  SLMs.  Optical  processing  architectures  developed  for  SLMs 
are  generally  inappropriate  for  use  with  SLRs.  While  promising,  the  unique  features 
of  SLRs  have  not  been  shown  definitively  to  offer  significant  advantages  over 
conventional  processors  or  more  importantly,  to  provide  potential  performance  that 
can  meet  the  Air  Force  avionics  requirements.  The  goals  of  this  program  are  to  1) 
select  or  develop  optical  processing  architectures  most  suitable  for  use  with  SLRs,  2) 
analyze  the  potential  performance  of  the  selected  architectures,  and  3)  demonstrate  the 
basic  operations  of  the  selected  SLR -based  optical  processors. 

1.2  Program  Structure 

The  SLR  Architecture  Study  program  is  composed  of  four  tasks,  beginning 
with  a  technology  survey  and  finishing  with  concept  demonstration  experiments  at  the 
Air  Force  Wright  Laboratory.  The  four  tasks  are  as  follows. 
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[ 

I  Task  1:  Identify  Air  Force  missions  requiring  high  speed  compact  processors  and  define 

the  computation  requirements.  Survey,  analyze  and  invent  optical  processing 
architectures  and  select  candidates  with  the  best  potential  for  satisfying  the  computation 
requirements. 

Task  2;  Analyze  selected  optical  processing  architectures  and  estimate  potential 
performance  based  on  projected  device  parameters.  Select  the  three  most  promising 
architectures  for  experimental  investigation.  Selection  criteria  include  potential 
performance,  SLR  requirements,  current  and  near  term  SLR  availability,  and  potential 
for  insertion  into  Air  Force  systems. 

Task  S;  Identify  key  operations  arid  component  requirements.  Perform  preliminary 
^  experiments  to  evaluate  the  performance  and  feasibility  of  implementing  the  three  optical 

I  processing  architectures  chosen  in  Task  2.  Select  two  optical  processing  architecture  for 

experimental  demonstration.. 

I 

Task  4;  Develop  and  perform  concept  demonstration  experiments  on  the  two  optical 

I  processing  architectures  selected  in  Task  3  at  the  Air  Force  Wright  Laboratory, 

1 

I  1.3  Spatial  Light  Rebroadcasters  (SLRs) 

The  generic  description  of  an  SLR  is  shown  schematically  in  Figure  1-1.  The 
input  light  field  with  data  coded  spatially  in  intensity  impinges  on  the  SLR  which  stores 
the  information.  The  SLR  emits  light  when  triggered  by  either  electrical  or  optical 
signals  encoded  spatially  with  a  second  set  of  data.  The  SLR  can  perform  basic 
processor  functions  such  as  memory,  summation  of  spatial  and  temporal  data, 
multiplication  of  two  data  sets  represented  by  the  incident  and  readout  signals,  and 
nonlinear  transformations  such  as  thresholding. 
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Figure  i-1.  Generic  Description  of  a  Spatial  Light  Rebroadcaster  (SLR) 
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SLMs  and  SLRs  both  operate  with  opacai  signals  with  information  encoded 
spatially  and  possess  read/write  storage  capabilities.  However,  there  are  substantial 
differences.  The  phase  and  coherence  of  the  input  light  field  are  not  preserved  by  an 
SLR.  Therefore,  information  in  the  input  and  output  beams  can  only  be  encoded  in 
the  intensity  of  the  radiation.  As  we  shall  show  in  Section  2,  this  restriction  impacts 
strongly  the  design  of  optical  architectures  that  utilize  SLRs. 

There  are  two  main  types  of  SLRs:  active  and  passive  SLRs.  While  both  can 
perform  the  generic  functions  described  above,  their  operating  characteristics  are 
fundamentally  different.  Optical  processing  architectures  that  optimally  utilize  these 
two  types  of  SLRs  therefore  cannot  be  the  same. 

An  active  SLR  can  be  considered  to  be  an  integrated  optoelectronic  device 
with  an  array  of  elements  as  illustrated  in  Figure  1-2.  Each  element  of  the  device  is 
composed,  at  a  minimum,  of  a  photodetector,  an  electronic  logic  and  control  unit  and 
an  emitter.  The  input  to  the  logic  and  control  unit  may  be  a  signal  from  another 
photodetector.  The  output  emission  is  determined  by  the  input  light  intensity  and  the 
electronic  control  signal  which  may  have  been  originated  by  an  optical  signal.  An 
example  of  a  series  of  operations  that  can  be  implemented  with  an  SLR  is  to  1) 
multiply  an  input  value  by  a  weight  represented  by  the  control  signal,  2)  threshold 
the  product  and  3)  output  the  binary  result.  This  can  be  achieved  by  encoding  the 
input  value  in  the  intensity  of  the  input  beam  and  the  weight  in  the  control  signal. 

The  control  unit  is  composed  of  an  amplifier  and  a  comparator  for  thresholding.  The 
control  signal,  which  can  be  generated  optically  via  a  photodetector,  controls  the  gain 
of  the  amplifier.  The  output  voltage  is  thus  proportional  to  the  product  of  the  input 
and  the  control  signals.  If  the  product  of  the  two  values  exceeds  a  preset  threshold, 
the  emitter  is  turned  on.  It  is  interesting  to  note  that  this  series  of  simple  operations 
is  the  heart  of  an  artificial  neural  network  which  will  be  described  in  Sections  2.4.3 
and  3.2.  With  a  two-dimensional  array  of  these  elements,  the  processing  function 
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Figure  1-2.  Functional  Model  of  an  Active  SLR 
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described  can  be  performed  in  parallel  by  all  the  elements  in  the  array,  achieving  a 
very  high  processing  throughput. 

Passive  SLRs  operate  by  a  very  different  principle  even  though  they  perform 
the  same  generic  functions  as  active  SLRs.  Instead  of  an  array  of  individual 
elements,  a  passive  SLR  typically  consists  of  a  uniform  layer  of  an  electo-optics 
material.  An  example  is  the  electron  trapping  phosphor  material  manufactured  by 
Quantex  [Lindmayer].  Incident  energy  at  a  short  wavelength  (e.g.,  Xj=  green)  is 
absorbed  by  the  material,  exciting  electrons  up  to  the  communication  band.  The 
electrons  then  fall  into  traps  where  they  are  stored.  When  the  material  is  radiated  by 
light  at  a  longer  wavelength  (e.g.,  X,=near  infrared),  the  trapped  electrons  are 
excited  out  of  the  trapping  level  and  fall  back  to  the  valance  band,  emitting  light  at 
wavelength  Xq  where  Xj  <  X^  <  X^.  The  number  of  trapped  electrons  is  determined 
by  the  product  of  the  intensity  of  the  input  radiation  and  the  number  of  trap  sites. 

The  intensity  of  the  output  emission  is  proportional  to  the  number  of  occupied  traps 
times  the  intensity  of  the  readout  beam.  The  products  of  two  arrays  of  values  can  be 
obtained  by,  for  example,  inputting  a  light  pattern  representing  the  values  of  one  of 
the  arrays  and  reading  the  SLR  out  with  a  light  pattern  corresponding  to  the  second 
array.  The  output  intensity  pattern  of  the  emitted  radiation  is  proportional  to  the 
product  of  the  two  arrays.  The  dynamic  behavior  of  a  passive  SLR  and  its  use  in 
incoherent  optical  processing  architectures  are  discussed  in  Section  3.5. 

The  performance  of  an  optical  processor  is  strongly  dependent  on  the 
performance  of  the  components  used  in  its  fabrication.  To  develop  and  evaluate 
optical  processing  architecture  utilizing  SLRs,  the  performance  parameters  achievable 
with  SLRs  must  first  be  defined.  In  Table  1-1,  the  projected  performances  of  active 
and  passive  SLRs  are  compared  with  that  of  an  SLM.  The  projected  performance  for 
the  active  and  passive  SLRs  are  based  on  data  from  AT&T  Bell  Laboratory  [Taylor] 
and  Quantex  Corporation  [Lindmayar],  respectively.  The  performance  measure  used 
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for  comparison  is  space-bandwidth  product  rate  (SBWP/T)  in  units  of  pixels/sec.  If  a 
single  operation  is  performed  each  time  the  device  goes  through  a  read/write  cycle, 
the  SBWP/T  is  also  the  system  throughput  in  terms  of  operations/sec.  The  potential 
performance  of  optical  processors  based  on  SLRs  are  orders  of  magnitude  higher  than 
those  implemented  with  SLMs. 

Table  1-1;  Projected  Performance  of  an  SLR 


Device 

SBWP 

(pixels) 

Frame  Time 
(sec) 

SBWP/T 
(pixels/ sec) 

Dynamic  Range 
(dB) 

Active  SLR 
(Opto-Electronics) 

lOhild^ 

10*® 

10^3 

30 

Passive  SLR 
(Phosphor) 

lO^^xlO^^ 

10*3 

10^^ 

50 

SLM 

10^x10^ 

10*2 

10* 

30 

There  are  other  considerations  in  evaluating  devices  besides  raw  performance 
such  as  SBWP/T.  The  features  and  limitations  of  the  two  types  of  SLRs  are 
summarized  in  Table  1-2. 
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Table  1-2:  Features  of  SLRs 


Device 


Features 


Limitations 


Active  SLR  High  speed,  programmability, 

(Opto-electronics)  nonlinearity,  memory, 

provides  gain,  ideal  for 
pipelined  architectures 


Relatively  small 
SBWP  (100x100), 
high  cost 


Passive  SLR 
(Phosphor) 


Large  SBWP,  memory,  low 
cost,  write  and  erase 
simultaneously  for  bias 
subtraction 


Weak  nonlinearity,  read 
out  destructive,  low  effic¬ 
iency,  relatively  slow,  read- 
write  at  different  wavelengths 


1 .4  Architecture  Study  Results 


The  choice  of  optical  processing  architectures  is  constrained  by  the  operating 
characteristics  of  SLRs.  The  optical  fields  used  to  write  on  or  read  out  an  SLR  can  be 
coherent  or  incoherent,  as  long  as  they  are  at  the  proper  wavelengths  but  the  output 
emission  is  always  incoherent.  The  output  data  are,  therefore,  represented  by  the  beam 
intensity  which  takes  on  only  positive  real  values.  Moreover,  an  SLR  does  not  simply 
modulates  the  intensity  of  the  input  light  field,  it  also  destroys  all  the  phase  information 
in  the  input  light  field.  Therefore,  SLRs  cannot  be  used  as  direct  replacement  for  SLMs 
in  optical  processors.  Even  with  incoherent  optical  processing  architectures,  the 
destruction  of  the  phase  information  limits  the  functions  an  SLR  can  perform.  For 
example,  an  SLR  cannot  be  used  as  an  aperture  mask  in  an  imaging  system.  The  SLR 
randomizes  the  phase  at  the  aperture  plane  and  no  image  can  form  at  the  output. 


The  active  and  passive  SLRs  also  have  distinct  characteristics  that  affect  the 
selection  of  optical  processing  architectures.  Being  passive  devices,  passive  SLRs  offer 
no  gain.  Moreover,  the  input  and  output  wavelengths  are  substantially  different. 
Together,  they  make  it  almost  impossible  to  cascade  SLRs  to  perform  sequential  or 


9 


iterative  operations.  On  the  other  hand,  active  SLRs  can  provide  optical  gain  and  the 
devices  can  be  easily  cascaded.  Active  SLRs  can,  therefore,  be  used  in  optical 
processing  architectures  that  perform  sequential  or  iterative  operation.  This  is  important 
because  the  SBWP  of  the  device  is  relative  small  (<  100  x  100).  Image  processing 
functions  cannot  be  performed  on  large  images  in  a  single  pass.  Partitioning  the  image 
and  the  processing  algorithm  is  often  necessary.  Compensating  for  its  small  SBWP  is 
the  very  fast  (nsec)  cycle  time  achievable  with  active  SLRs.  The  time  constants  for  the 
fluorescence  and  the  stimulated  photoluminescence  are  relative  long  which  limits  the 
cycle  time  of  a  processor  utilizing  passive  SLR.  However,  the  space-bandwidth  product 
(SBWP)  provided  by  a  passive  SLR  can  be  very  large.  The  spatial  resolution  of  a 
phosphor  based  SLR  can  be  as  high  as  40  Ip/mm.  With  a  25mm  x  25mm  sample,  the 
number  of  resolvable  elements  or  SBWP  is  1000  x  1000.  Optical  processing 
architectures  utilizing  passive  SLRs  must  not  demand  fast  cycle  time  and  should  take  full 
advantage  of  their  large  SBWP. 

A  pre-selection  was  performed  that  resulted  in  seven  candidates  for  evaluation. 
The  unique  operating  characteristics  of  active  and  passive  SLRs  were  taken  into  account 
in  the  preliminary  selection  of  optical  processing  architectures.  The  selection  was  based 
primarily  on  the  functionality,  versatility  and  practicality  of  the  architecture.  The 
architectures  chosen  for  study  were  1)  Scanning  Correlator,  2)  Interferometric  processor, 
3)  OTF  synthesis  Optical  Preprocessor,  4)  Artificial  Neural  Network,  5)  Quadratic 
Processor,  6)  Morphological  Processor  and  7)  Multispectral  Optical  Processor. 

The  scanning  correlator  [Lee]  utilizes  the  capability  of  a  passive  SLR  to  store 
image  information  and  produce  an  output  that  is  proportional  to  the  product  of  the  stored 
image  data  and  the  image  data  encoded  in  the  read  beam.  The  interferometric  processor 
[Tai,  Aleksoff]  performs  Fourier  transformation  on  the  intensity  distribution  of  an 
incoherent  input  field.  The  large  dynamic  range  provided  by  a  passive  SLR  is  used  with 
its  simultaneous  write-erase  capability  to  enhance  the  output  of  an  interferometric 
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processor  which  is  characterized  by  a  high  bias.  The  same  features  are  employed  in  the 
OTF  synthesis  optical  processor  [Rhodes]  to  perform  pre-detection  spatial  filtering  of 
image  data.  The  artificial  neural  network  [Lippman]  ,  the  quadratic  processor  [Rugh] 
and  the  morphological  processor  [Sternberg]  perform  nonlinear  imaging  processing 
operations.  They  all  require  sequential  sum  of  products  operations  which  are  particularly 
suited  for  active  SLRs.  The  multispectral  optical  processor  takes  advantages  of  the  fact 
that  different  wavelengths  are  used  to  write  on  and  erase  data  from  a  passive  SLR  to 
perform  real  time  pre-detection  enhancement  of  the  image  signal-to-clutter  ratio. 

Of  the  seven,  five  were  down  selected  for  further  analyses.  The  down  selection 
was  based  primarily  on  functionality,  versatility  and  practicality.  They  are  the  artificial 
neural  network,  the  quadratic  processor,  the  morphological  processor,  the  interferometric 
processor  and  the  OTF  synthesis  optical  preprocessor.  Reasons  for  their  selection  are 
described  below. 

Unlike,  for  example,  a  Fourier  transform  based  processor,  an  artificial  neural 
network  (ANN)  operates  from  a  low  level  which  makes  it  the  most  versatile  of  the 
architectures  studied.  It  can  be  used  to  implement  nearly  all  the  functional  elements  in 
ATC.  The  basic  operation  required  is  sum  of  products  and  thresholding  which  can  be 
performed  very  efficiently  with  an  optical  processor.  In  addition,  the  massive  fan-outs 
in  an  ANN  architecture  can  be  accomplished  more  easily  with  optical  interconnects  than 
with  electrical  wires. 

A  quadratic  processor  performs  pixel-by-pixel  statistical  target  detection  on  the 
target  scene.  It  utilizes  local  spatial  variations  (reflectance  or  emittance)  as  a 
discriminant  between  targets  and  clutter.  The  processing  architecture  can  utilize  the 
incoherent  sensor  image  directly  as  the  input.  It  allows  the  quadratic  processor  to  bypass 
the  limitations  of  optical  to  electrical  and  electrical  to  optical  converters,  making  it 
particularly  useful  as  a  preprocessor.  In  addition,  the  nonlinear  operations  performed  by 
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a  quadratic  processor  provide  processing  capabilities  not  available  with  conventional 
linear  optical  systems. 

The  morphological  processor  is  also  a  neighborhood  processor  capable  of 
nonlinear  processing  functions.  Instead  of  matching  to  the  overall  shape  of  a  target,  a 
morphological  processor  performs  ATC  by  extracting  features  that  define  a  target.  Such 
a  process  tends  to  be  more  robust  than  matched  filtering  or  template  matching  whose 
performance  can  be  adversely  affected  by  changes  in  aspect,  lighting  and  operating 
condition.  Morphological  processing  is  typically  implemented  electronically  using  a 
recirculating  pipeline  architecture  to  minimize  the  size  of  the  processor.  With  the 
inherent  parallelism  of  an  optical  processor,  the  neighborhood  operation  can  be 
performed  simultaneously  for  all  pixels  in  the  image. 

The  last  two  are  diffiaction  based  incoherent  optical  processors  whose 
architectures  are  well  known.  Diffraction  based  systems  provide  the  largest  space- 
bandwidth  product,  making  them  attractive  for  wide  area  search  applications.  The 
difficulty  has  been  with  the  bias  which  could  easily  overwhelm  the  signal  at  the  output. 
With  both  architectures,  the  role  of  the  SLR  is  bias  reduction.  Since  the  principles  of 
these  two  types  of  incoherent  optical  processors  are  well  established,  the  discussion  in 
Section  3  will  concentrate  on  the  use  of  a  passive  SLR  for  bias  reduction. 

The  scanning  correlator  was  not  chosen  because  of  its  limited  capability  (the 
reference  function  must  be  real  and  positive)  and  relatively  slow  speed  due  the  serial 
nature  of  the  scanning  operation.  The  muldspectial  optical  preprocessor  did  not  survive 
the  down  selection  because  the  laser  power  required  for  flood  illumination  may  be  too 
high  to  be  practical  in  view  of  the  sensitivity  of  available  passive  SLRs  .  Moreover, 
discrimination  between  different  sets  of  target  and  clutter  may  involve  different 
combinations  of  wavelengths.  To  reprogram  the  preprocessor  will  require  a  change  in 
the  SLR  material  to  one  with  different  input  and  readout  wavelengths. 
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1.5  Performance  Evaluations 


In  this  section,  the  predicted  performances  of  the  selected  optical  processing 
architectures  are  presented.  The  purpose  of  the  performance  evaluation  is  to  compare 
the  performance  of  the  optical  processors  with  those  of  their  electronics  counteiparts  in 
performing  similar  ATR/C  algorithms.  It  is  not  the  goal  of  this  project  to  evaluate  the 
effectiveness  of  various  ATR/C  approaches  and  algorithms,  Hierefore,  the  performance 
was  evaluated  in  terms  of  processing  throughput  instead  of  the  probabilities  of  detection 
and  false  alarms. 

1.5.1  Optical  Artificial  Neural  Network 

Optical  processing  generally  has  less  accuracy  than  electronic  digital  processing 
but  greater  speed.  In  other  ERIM  work,  a  study  was  done  to  determine  the  accuracy 
required  for  artificial  neural  network  computations.  The  problem  chosen  was  that  of 
determining  terrain  type  (forest,  grass,  soil)  from  airborne  sensor  imagery  of  the  ground 
in  five  wavelength  bands  in  the  visible,  near  infrared,  and  short  wave  infrared.  A 
Kohonen  self-organizing  network  was  successfuUy  developed  for  this  purpose.  The 
network  has  five  inputs,  three  nodes,  and  three  outputs  corresponding  to  the  terrain 
classification.  The  network  was  trained  with  floating  point  computation.  The  network 
was  then  used  to  classify  the  input  data  with  varying  accuracy  in  the  input  data  and 
weights.  The  result  is  that  floating  point  performance  is  maintained  down  to  5-bit 
accuracy  in  the  data  and  weights.  Although  this  is  only  a  single  test,  it  was  assumed  that 
6-bit  accuracy  is  sufficient  for  artificial  neural  network  computations  during  use,  but  not 
during  training. 

In  Section  2.4.4,  five  optical  neural  network  architectures  are  described.  To 
perform  the  performance  evaluation,  a  cylindrical  optics  architecture  and  a  planar 
architecture  with  fixed  weights  were  chosen.  Two  versions  of  the  cylindrical  optics 
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architecture  were  designed  and  analyzed  using  a  lens  design  program.  The  following 
parameters  were  used  in  the  point  design  for  a  neural  network  processor  with  cylindrical 
optics. 

Input:  100  laser  diodes  at  1  mm  spacing  and  1  MHz  operation  rate,  Mask:  100  by  200 
1mm  by  250  micron  pixels 

Output:  200  photodiodes  at  250  micron  spacing  and  1  MHz  operation  rate. 

This  leads  to  the  following  performance  characteristics: 

Volume:  L  by  W  by  H  =  20  cm  by  10  cm  by  5  cm  =  1000  cm^  , 

Input  data  rate:  100  x  1  MHz  =  10*  data  values/sec 
Computation  rate:  100  x  200  x  1  MHz  =  2  x  10^*^  operations/sec. 
Computations/Volume:  2  x  10^  operations/sec/cm^. 

Power  (for  1  %  accuracy!:  100  x  40  mW  =  4  W 

For  the  planar  architecture,  the  following  parameters  were  used: 

Input:  100  laser  diodes  at  100  micron  spacing  and  1  MHz  operation  rate 
Mask:  100  by  200  100  micron  by  50  micron  pixels 

Output:  200  linear  photodiodes  at  50  micron  spacing  and  1  MHz  operation 
rate 

This  leads  to  the  following  characteristics: 

Volume:  L  by  W  by  H  =  6  cm  by  1  cm  by  1  cm  =  6  cm^- 
Input  data  rate:  100  x  1  MHz  =  10*  data  values/sec 
Computation  rate:  100  x  200  x  1  MHz  =  2  x  10  operations/sec 
Computations/cm^:  3  x  10^ 

Power  (for  1%  accuracy):  100  x  40  mW  =  4  W 
This  second  architecture  is  superior  to  the  first  in  computation/cm^  as  desired. 
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1.5.2  Optical  Quadratic  P»*ocessor 


The  optical  implementation  of  a  quadratic  processor  geared  toward  automatic 
target  recognition  (ATR)  was  considered.  This  processor  implements  the  likelihood  ratio 
detector  which  is  used  extensively  in  ATR  activities.  Our  analysis  has  shown  that  the 
optical  implementation  of  the  quadratic  processor  has  a  number  of  distinct  advantages 
over  their  electronic  (and  other  optical  architectures)  counterparts,  most  notably  in 
throughput  rate  and  density.  The  number  or  operations  performed  on  each  input  pixel 
is  summarized  as: 

Multiplication  by  mask: 

Lens  summation  (8  adds  x  9  real.) 

Multiplication  by  neighborhood 

Final  summation 


Total  170  ops/input  pixel 


81  multiplications 
72  additions 
09  multiplications 
08  additions 


'  For  input  image  sizes  of  5(X)  x  500  pixels,  the  system  throughput  rate  is: 

^  1.4  Gops/sec  (SLR  response  time  =  30  msec) 

42.5  Gops/sec  (SLR  response  time  =  1  msec) 

42,500  Gops/sec  (SLR  response  time  =  1  fisec) 

I 

I  , 

In  addition,  our  analysis  showed  that  the  system  size  is  approximately  41,000  cm-’ 

j  and  a  prime  power  requirement  of  750  Watts.  The  throughput  rate  per  unit  volume  is 

then  1  Gop/sec  cm^  and  the  throughput  rate  per  unit  power  is  57  Gop/sec  W  (SLR 
response  time  =  1  ;xsec).  The  signal-to-noise  ratio  of  the  system  was  shown  to  be  at 
approximately  17dB.  These  specifications  make  the  optical  implementation  an  extremely 
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viable  architecture.  In  addition,  it  should  be  noted  that  this  architecture  requires  no 
optical/electronic/optical  transduction,  it  operates  directly  on  the  imaging  sensor 
pre-detected  output. 

1,5.3  Optical  Morphological  Processor 

The  ideal  processor  architecture  for  morphological  processing  in  terms  of 
performance  is  the  parallel  full  array.  All  image  pixels  are  transformed  simultaneously, 
providing  a  tremendously  high  throughput.  Its  implementation,  unfortunately,  is  not 
feasible  with  current  microelectronics  fabrication  technology.  The  inherent  massive 
parallelism  of  an  optical  processor,  however,  may  make  it  possible  to  implement  a 
parallel  full  array  in  a  reasonably  compact  package,  resulting  in  orders  of  magnitude 
improvement  in  system  throughput. 

The  heart  of  an  optical  morphologic  image  processor  is  the  computation  unit 
which  is  composed  simply  of  an  input  SLR,  an  imaging  lens,  a  programmable  pupil  mask 
and  an  output  SLR  as  described  in  Section  3.4.  This  basic  module  can  be  cascaded  and 
arranged  in  a  feedback  architecture.  The  data  circle  back  after  passing  through  and 
processed  by  the  N  stages.  The  processing  throughput  of  such  a  processor  is  maximized 
when  the  time  required  to  alter  the  transmittance  of  the  SLMs  is  matched  to  the 
processing  time  through  the  N  stages.  If  for  example,  the  switching  time  of  the  SLR  is 
1  nsec  and  N  =  50,  the  SLM  and  the  SLR  logic  must  be  programmable  within  50  nsec 
to  keep  up.  Otherwise,  the  processing  speed  must  be  slowed  down  or  the  number  of 
processing  stage  has  to  be  increased. 

Let  us  assume  that  the  structure  element  is  composed  of  3  x  3  neighborhood 
pixels,  a  single  transformation  will  require  a  minimum  of  nine  multiplies  and  one 
summation  for  a  total  of  ten  arithmetic  operations.  If  the  space-bandwidth  product 
(SBWP)  of  the  SLRs  is  256  x  256  pixels,  then  with  the  speed  assumed  in  Table  1  for  the 
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active  SLR  devices,  the  processing  speed  of  the  optical  morphological  processor  will  be 
256^  X  10  ops/1  nsec  =  6  x  10*'*  ops/ sec. 

A  rough  estimate  of  the  processor  size  is  about  5cm  x  5cm  x  10cm  for  each 
optical  stage.  The  optics  of  a  10  stage  system  will  occupy  about  2500  cm^.  Adding 
another  10,000  cm^  for  the  control  and  driving  electronics,  the  total  processor  volume 
is  about  12,500  cm^.  The  processor  throughput  per  unit  volume  is  then  equal  to  5  x  10^° 
operations/ sec/cm^ . 

If  we  use  a  more  modest  and  realistic  processor  size  and  the  throughput  becomes 
SLM  switching  time  limited,  then  with  a  SLM  switching  speed  of  1  fisec,  the  processing 
speed  is  lowered  to  256^  x  10  x  10  ops/  1  nsec  ~  6.5  x  10^^  ops/sec.  The  processing 
speed  per  unit  volume  achieved  with  these  rather  conservative  parameters  is  then  equal 
to  5.2  X  10®  ops/sec/cm^  which  is  still  very  high. 

1.5,4  Interferometric  Processor 

The  interferometric  processor  can  be  implemented  with  a  different  architecture. 
Since  the  optical  input  to  an  interferometric  processor  is  incoherent,  the  natural  target 
scene  can  theoretically  be  used  directly.  However,  the  amount  of  light  available  in  a 
natural  scene  at  the  write  wavelength  of  an  SLR  may  not  be  enough  to  write  on  the  SLR 
at  a  high  rate.  To  perform  image  processing  at  high  speed,  the  write  beam  must  have 
sufficient  amount  of  optical  power.  One  optical  architecture  considered  is  based  on 
acousto-optics  scanners/modulators.  Let  the  aperture  time  be  t,  N  be  the  number  of 
pixels  on  a  carrier  in  the  A-0  cell  and  be  the  space-bandwidth  product  of  the 
processor  output.  The  processing  speed  of  the  processing  systeni  is  then  equal  to  M^N/r. 
As  an  example,  with  a  Crystal  Tech  4075  A-0  modulator,  the  carrier  frequency 
4  =  75MHz,  the  bandwidth  BW  =  50MHz,  t  =  SOftsec  and  M  =  N  =  4000.  The 
processing  speed  is  then  equal  to  1.25  x  10*^  op/sec.  The  optics  in  the  interferometric 
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processor  should  occupy  about  8700  cm^  Including  all  the  driving  electronics,  the 
overall  processor  size  is  estimated  to  be  approximately  3  x  10^  cm^.  The  predicted 
system  performance  of  the  optical  interferometric  processor  per  unit  volume  is  about 
4.2  X  10®  ops/sec/cm^. 

1.6  Conclusion 

The  Spatial  Light  Rebroadcaster,  particularly  of  the  active  type,  can  potentially 
be  a  powerful  device  that  can  serve  as  the  heart  of  a  compact  high  speed  processor.  The 
devices,  however,  are  still  in  a  very  early  developmental  stage  and  they  require 
significant  amount  of  further  developn  mt  bw.’ore  they  can  be  competitive  in  optical 
processing  architectures  such  as  those  described  in  this  report. 

Passive  SLRs  such  as  those  implemented  with  electron-trapping  materials,  exist 
today.  Some  of  these  materials  were  developed  for  wavelength  down-conversion  to 
visualize  near  inftared  radiation  and  they  are  commercially  available.  The  performance 
of  these  passive  SLR  materials  and  devices,  however,  require  substantial  improvement 
in  several  of  areas  to  make  them  competitive. 

1)  The  slow  temporal  response  of  the  passive  SLR,  particularly  in  erasure,  limits  the 
cycling  rate.  The  throughput  achievable  is  too  slow  to  be  competitive  at  this  time. 

2)  Compounding  the  problem  of  low  cycling  rate  is  the  low  optical  efficiency.  The 
output  is  so  dim  that  the  output  must  be  integrated  over  a  significant  amount  of  time 
to  gather  enough  photons  to  provide  the  needed  signal  dynamic  range. 

3)  The  erasure  is  often  incomplete  unless  very  strong  light  or  heat  is  used.  The  need 
for  a  powerful  source  for  rapid  and  complete  erasure  impacts  negatively  on  power 
consumption. 

One  solution  to  the  problem  may  be  to  develop  an  SLR  that  emits  light 
directionally  (current  devices  radiate  isotopically,  over  4t  radian).  Improving  the  optical 
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efficiency  would  allow  the  use  of  a  thin  layer  of  rebroadcasting  material  and  improve  the 
cycling  speed  of  the  device. 


More  serious  are  some  of  the  inherent  characteristics  of  passive  SLRs  which  limit 

their  usefulness. 

1)  The  readout  is  destructive.  The  material  requires  constant  refreshing  to  keep  the  data 
stored  in  the  device.  A  trade  off  between  output  brightness  and  the  number  of 
number  of  times  the  stored  in  formation  can  be  readout  is  required. 

2)  The  input  and  readout  wavelengths  are  different  which  precludes  the  cascading  of 
devices  to  perform  sequential  operations  even  if  adequate  optical  efficiency  can  be 
achieved. 

3)  The  nonlinearities  exhibited  by  passive  SLRs  are  weak  and  they  can  not  be  easily 
changed.  The  type  of  operation  that  can  be  performed  is  therefore  restricted. 


With  these  inherent  limitations,  passive  SLR  devices  are  less  likely  to  have  a 
significant  impact  on  the  optical  processor  development. 


Active  SLR  devices  have  the  inherent  flexibility  and  power  to  be  a  significant 
player  in  the  future  development  of  compact  high  speed  processing  systems.  They  may 
be  utilized  as  interconnects  in  an  electronic  processor,  or  as  the  processing  elements  in 
an  hybrid  electronic/optical  processor.  The  programmable  gain  and  nonlinearity 
provided  by  the  device  are  particularly  crucial  to  many  optical  computing  architecture. 
The  development  of  these  devices,  however,  is  still  in  an  early  stage.  Specific  area  that 
requires  further  development  includes  the  following. 


1)  Space-Bandwidth  Product.  The  advantage  offered  by  an  optical  processor  is  the 
massive  parallelism  of  the  computation.  This  advantage  can  be  realized  only  if  the 
space-bandwidth  product  of  the  input  and  output  devices  are  sufficiently  large. 
Devices  being  fabricated  at  this  time  are  very  small.  The  manufacturing  technology 
to  fabricate  a  large  array  with  acceptable  cost  and  yield  remains  to  be  developed. 

2)  Packing  Density.  The  most  attractive  promise  of  optical  processing  is  high  speed 
processing  in  a  small  physical  package  with  low  power  consumption.  To  fulfill  this 


19 


promise,  the  large  space-bandwidth  product  must  be  accomplished  in  a  small  package 
that  draws  little  power.  Therefore,  the  device  size  must  be  small  and  the  packing 
density  must  be  very  high.  Considering  that  each  element  in  an  active  SLR  consists 
of  a  detector,  a  signal  conditioner  and  an  emitter,  a  3-dimensional  structure  is  likely 
to  be  required  to  achieve  the  density  desired. 

3)  Addressing  Schemes.  To  maintain  a  high  throughput,  particularly  with  a  pipelined, 
recirculating  processing  architecture,  an  efficient  means  must  be  available  to  address 
and  program  the  elements  in  the  SLR  in  parallel. 

Optical  processing  and  computing  approaches  typically  fall  into  one  of  two 
categories.  The  optical  processor  either  performs  complete  high  level  operations  such 
as  correlation  for  matched  filtering  or  it  is  designed  to  perform  low  level  logic  operations 
that  emulate  electronic  processors.  Performing  the  bulk  of  the  ATR  functions  optically 
will  restrict  the  ATR  algorithms  to  linear  filtering  operations  which  severely  limits  the 
power  and  robustness  that  can  be  accomplished.  On  the  other  hand,  optics  has  limited 
success  in  challenging  the  well  developed  and  entrenched  electronics  technology  in 
performing  low  level  logic  operations.  The  optical  processing  architectures  presented  in 
this  report  offer  an  alternative  approach  where  optics  and  electronics  share  the  burden, 
each  doing  what  it  does  best.  Such  a  hybrid  processing  architecture  has  the  potential  to 
combine  the  speed  and  efficiency  provided  by  optical  processors  with  the  flexibility  and 
programmability  offered  by  electronic  processors.  Five  promising  optical  processing 
architectures  were  developed  and  analyzed.  Based  on  the  projected  performance  of  the 
passive  and  active  SLRs,  the  performance  of  the  five  processing  systems  were  estimated 
and  the  results  are  summarized  in  Table  1-3. 
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Table  1-3.  Summary  of  Predicted  Performance  of  Five  Optical  Processor  Architectures 


Architecture 

Device 

Rate/Vol. 

(ops/sec/cm3) 

Hm 

Neural  Network 

ANN  1 

Active 

1x10^ 

2 

1X103 

ANN  2 

Active 

3x109 

33 

6x10° 

Planar  ANN 

Active 

2x10io 

400 

0.5x109 

Quadratic  Processor 

Active 

1x109 

18 

41x103 

Morphological 

Active 

6x1 0’O 

33 

3x103 

OTF  Synthesis 

Passive 

1x106 

33 

3x102 

Interferometric  Processor 

Passive 

4x108 

20 

9x103 
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2.0  TASK  2;  ARCHITECTURE  STUDY 


The  goal  of  this  task  is  to  identify  several  promising  optical  processing 
architectures  that  can  take  advantages  of  the  unique  features  of  SLRs,  evaluate  their 
applicability  to  Air  Force  problems,  perform  preliminary  analyses  of  their  potential 
performance,  and  select  five  candidates  for  in  depth  evaluation  in  Task  2. 

2.1  Architecture  Study  Methodology 

The  methodology  used  for  the  architecture  study  is  summarized  in  Figure  2.1-1. 
The  study  began  with  an  examination  of  Air  Force  applications  and  identified  automatic 
target  classification  (ATC)  as  a  primary  application  for  high  speed  processors.  ATC  was 
first  broken  down  into  functional  elements.  The  processing  algorithms  for  these  ATC 
functional  elements  were  surveyed  and  the  mathematical  operations  required  were 
identified.  Next,  optical  processing  architectures  that  can  be  used  to  perform  ATC 
functions  were  assessed.  The  assessment  started  with  existing  optical  processing 
architectures  that  may  be  utilized  to  implement  the  ATC  algorithms.  Each  architecture 
was  analyzed  to  determine  if  and  how  processing  performance  can  be  improved  by  the 
use  of  SLRs.  Based  on  the  insight  gained  by  the  assessments,  modifications  of  the 
existing  architectures  were  made  and  new  architectures  were  developed  to  better  utilize 
the  characteristics  of  SLRs.  The  architecture  selection  was  influenced  from  the  top  by 
the  potential  of  the  optical  processing  architecture  to  satisfy  specific  needs  of  the  Air 
Force  and  from  the  bottom  by  the  availability  of  an  SLR  with  the  required  performance 
characteristics. 

2.2  ATC  Application  Requirements  and  Functional  Elements 

An  automatic  target  classifier  assigns  target  categories  with  associated  confidence 
measures  to  the  detected  targets.  The  level  of  classification  is  dependent  on  the  mission. 
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Figure  2.1-1.  Architecture  Study  Methodology 
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The  classification  can  range  from  coarse  (e.g.,  tree  clutter  versus  vehicle)  to  very  fine 
(e.g.,  an  M-1  tank  with  Allied  markings  versus  an  M-1  tank  with  markings  of  the 
adversary).  ATC  typically  begins  with  interest  point  location  which  defines  the 
probability  of  an  area  having  a  target.  It  could  be  based  on  terrain  information  or  target 
attributes  such  as  target  brighmess  and  contrast  or  multispectral  signature.  Areas 
identified  as  points  of  interest  are  then  segmented  further  into  regions  such  as  vehicles, 
trees,  roads,  houses,  hangers,  etc.  Segmentation  may  be  based  on  the  overall  dimension 
of  the  region,  surface  roughness  or  spectral  properties.  If  required,  even  finer  features 
in  the  segmented  areas  can  be  extracted.  Features  of  interest  could  include  wheels, 
tracks,  gun  barrels,  raised  deck  or  recessed  bay,  antennae,  camouflage  and  other 
markings.  The  final  step  is  to  classify  or  assign  target  categories  with  associated 
confidence  measures  to  the  segmented  regions  in  the  image,  identifying  them  as  clutter, 
friendly  targets,  and  hostile  targets  of  high  or  low  value.  The  functional  elements  in  a 
automatic  target  classifier  [ATRWG]  are  summarized  graphically  in  Figure  2.2-1. 

The  ultimate  figures  of  merit  for  an  ATC  are  the  probabilities  of  detection  and 
false  alarm.  The  goal  of  this  project,  however,  is  not  to  develop  or  optimize  TC 
algorithms.  The  goal  is  to  study  the  feasibility  of  implementing  existing  ATC  algorithms 
with  optical  processors  that  employ  SLRs.  The  figure  of  merit  that  will  be  used  is  how 
well  the  optical  processor  can  perform  the  ATC  algorithms. 

As  a  first  order  estimate  of  the  processing  throughput  requirement  for  ATC, 
consider  two  imaging  systems:  1)  a  pushbroom  imager  with  10^  linear  pixels  providing 
1  ft  ground  resolution  on  a  airborne  platform  travelling  at  600mph,  and  2)  a  focal  plane 
imager  with  512  x  512  pixel  refreshing  at  30  frames/sec.  For  both  sensors,  the  pixel  rate 
is  10^  pixels/sec.  Let  us  assume  that  it  requires  10  operations  /pixel  to  perform  a  simple 
detection  algorithm.  The  required  system  throughput  is  then  10*  ops/sec  which  is  within 
the  range  of  state-of-the  art  all  electronic  processors.  However,  simple  detection 
algorithms  are  generally  not  robust  enough  for  Air  Force  applications.  They  tend  to 
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Figure  2.2-1.  Functional  Elements  in  an  Automatic  Target  Classifier 
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degrade  substantially  with  target  and  clutter  variability.  More  robust  adaptive  ATC 
algorithms  typically  require  >  >  10  operations/pixel  to  implement.  The  processor 
throughput  requirement  is,  therefore,  in  the  range  of  10^  ops/sec. 

Optical  processors  are  most  effective  as  special  purposed  "hardwired"  processing 
units  where  their  strengths  are  optimally  utilized  and  its  weaknesses  are  circumvented. 
Therefore,  it  is  generally  more  practical  and  efficient  to  implement  one  or  more 
functional  elements  of  ATC  by  an  optical  processor  instead  of  an  entire  automatic  target 
classifier.  In  the  following  section,  incoherent  optical  processing  techniques  are  first 
reviewed.  Candidate  processor  architectures  are  then  described  in  Section  2.4  and  the 
ATC  functions  that  can  be  performed  by  each  architecture  are  discussed. 

2.3  Incoherent  Optical  Processing  Techniques  Overview 

An  SLR  does  not  modulate  the  input  or  the  readout  radiation,  it  absorbs  the  input 
radiation  and  re-emits.  There  is  no  correlation  between  the  phases  of  the  input,  the 
readout  and  the  output  light  waves.  Moreover,  the  output  radiation  is  spatially  incoherent 
and,  in  most  cases,  temporally  incoherent  as  well.  Optical  processing  architectures 
utilizing  SLMs  must  therefore  employ  incoherent  optical  processing  techniques.  In  this 
section,  basic  operations  that  can  be  accomplished  with  incoherent  optical  processing 
techniques  using  an  SLR  are  described,  starting  from  low  level  operations  such  as 
addition  and  multiplication  to  high  level  operations  such  as  Fourier  transformation.  The 
optical  processing  architectures  described  later  all  make  use  of  one  or  more  of  these 
techniques. 

2.3.1  Arithmetic  Operations 

Addition  and  multiplication  are  natural  operations  for  incoherent  optical 
processing  with  an  SLR.  Processing  algorithms  requiring  only  additions  and 
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multiplications  are  most  efficiently  performed.  Other  operations  such  as  division  are  also 
possible  but  requires  more  complexity  or  steps. 


Addition,  yx.y)  =  Ii(x,y)+l2(x,y): 

Passive  SLR  -  The  number  of  the  excited  and  trapped  electrons  is  proportional  to  the 
total  absorbed  energy.  With  two  input  light  patterns  incident  on  the  SLR  simultaneously, 
the  number  of  trapp^  electrons  is  given  by  the  sum  of  the  intensities.  With  the  storage 
capability  of  the  SLR,  the  summation  can  also  be  performed  by  two  input  patterns 
incident  on  the  device  sequentially.  The  ability  to  sum  sequential  inputs  is  the  primary 
benefit  of  SLR  over  a  conventional  detector. 

Active  SLR  -  Data  can  be  stored  as  charges  in  capacitors  and  summation  can  be 
performed  serially  or  in  parallel  as  with  a  passive  SLR. 

MultipUcation,  Io(x,y)  =  Ii(x,y)l2(x,y): 

Passive  SLR  -The  light  pattern  Ii(x,y)  is  input  to  the  SLR  and  stored.  The  SLR  is  then 
readout  with  light  pattern  l2(x,y).  The  emitted  output  light  pattern  is  proportional  to 
Ii(x,y)l2(x,y)  provided  that  the  number  of  trapped  electrons  depleted  by  the  readout 
beam  is  small  compared  to  the  total  number  of  trapped  electrons. 

Active  SLR  -  Multiplication  can  be  achieved  by  using  one  input  value  to  control  the  gain 
of  the  amplifier.  The  amplified  output  is  then  proportional  to  the  product  of  the  input 
value  and  the  stored  value  controlling  the  gain. 

Contrast  Reversal,  Io(x,y)  =  C  -  Ii(x,y) 

Passive  SLR  -  A  uniform  pattern  is  input  and  store,  then  read  out  with  Ii(x,y).  What 
remains  stored  in  the  trap  sites  of  the  material  is  Cj  -  li(x,y).  Reading  the  SLR  out 
again  with  a  uniform  beam  produces  Io(x,y)  =  C2[Ci  -  Ii(x,y)]. 

Active  SLR  -  Contrast  reversal  can  be  accomplished  with  an  inverting  amplifier. 

Subtraction,  Io(x,y)  =  Ii{x,y)  -  l2(x,y) 

Passive  SLR  -  Contrast  reversal  is  first  performed  on  the  input  pattern  Ii(x,y)  as 
described  above  and  obtain  [Cj  -  Ii(x,y)].  The  input  second  light  pattern  l2(x,y)  is  then 
read  out  with  a  uniform  beam.  The  emitted  output  is  given  by  Io(x,y)  =  Cl  +  l2(x,y)  - 
Ii(x,y). 


Active  SLR  -  Charges  stored  in  capacitors  can  be  depleted  by  the  desired  amount  to 
perform  subtraction. 

Division,  Io(x,y)  =  l2(x,y)  /  Ijfx.y) 

Passive  SLR  -  Division  can  be  performed  with  a  device  having  a  nonlinear  input/output 
transfer  characteristic  described  by  Iq  =  above  the  toe  region  where  7=1.  It  is  a 
natural  characteristic  of  many  materials  to  have  a  negative  input-output  transfer 
characteristic  which  gradually  reaches  saturation.  The  material  acts  as  an  invertor,  i.e. , 
IJ(x,y)  =  1/Ii(x,y)  for  input  values  that  are  >0,  Reading  out  the  SLR  widi  a  light 
pattern  l2(x,y),  we  have  Io(x,y)  =  l2(x,y)  Ii(x,y)  =  l2(x,y)  /  Ii(x,y). 

Active  SLR  -  The  same  nonlinearity  can  be  used  to  implement  division. 


We  note  that  the  operations  are  performed  in  parallel  on  all  elements  of  the  array.  If  we 
have  a  large  number  of  elements  in  the  array,  the  computation  rate  in  terms  of 
operation/ sec  can  be  very  high. 


2.3.2  Higher  Level  Operations 


Higher  level  operations  can  be  implemented  by  combining  basic  arithmetic 
operations.  However,  optical  phenomena  can  also  be  used  to  implement  higher  order 
operations  directly.  Coherent  optical  processors,  for  example,  make  extensive  use  of  the 
Fourier  transform  property  of  a  lens.  Equivalent  operations  can  also  be  performed  with 
incoherent  optical  processors. 

2.3.2.1  Matrix  Multiplication 


The  most  well  known  higher  order  operations  accomplished  by  combining  basic 
arithmetic  operations  are  vector-matrix  multiplications,  Ajy  =  and  matrix- 

matrix  multiplications,  =  Bi^C|q4  .  The  elements  in  the  output  vector  in  a 
vector-matrix  multiplication  are  given  by: 
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M 

^n~  53  ^rrPmn 
m-\ 

% 

For  matrix-matrix  multiplication,  the  output  matrix  is  equal  to: 

K 

^mn  ~  52  ^mk^kn 
k=l 


(i) 


(2) 


Both  involve  sum  of  products  operations  requiring  only  additions  and  multiplications 
which  can  be  performed  very  efficiently  by  an  optical  processor.  Sum  of  products  is  the 
heart  of  many  processing  algorithms  including  Fourier  transformation  and  spatial 
frequency  filtering,  convolution,  correlation,  quadratic  processing  and  artificial  neural 
networks. 

2.3.2.2  Fourier  Transformation 

Fourier  transformation: 

0&  00 

F(u,  v)  =  ^[f(x,  y)]  =  I  I  f(x,  y)  dxdy  (3) 

^oo  -oo 

or  its  discrete  form: 

F(u.  V)  =  iT  [f(x„,  y„)]  =  I;  f:  f(x„,  y„)  W 

m*-*  n=-06 

can  be  implemented  as  vector-matrix  multiplications.  The  Fourier  transform  kernel  is 
complex  (i.e. ,  with  real  and  imaginary  parts).  Even  if  the  input  is  composed  only  of 
positive  real  values,  the  processor  must  still  be  able  to  perform  complex  multiplications 
and  additions.  With  an  incoherent  optical  vector-  matrix  or  matrix-matrix  multiplier 
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which  can  only  represent  positive  real  values  via  light  intensities,  the  computations  have 
to  be  carried  out  with  a  minimum  of  three  parallel  channels. 


Fourier  transformation  can  be  more  efficiently  accomplished  by  taking  advantage 
of  the  properties  of  light  propagation.  With  coherent  light,  there  is  a  Fourier  transform 
relationship  between  the  complex  amplitudes  of  the  fields  at  two  widely  separated  planes. 
The  long  propagation  distance  for  Fraunhofer  diffraction  can  be  significantly  shortened 
with  the  use  of  a  lens  which  performs  the  coherent  integration  operation  in  the  Fourier 
transformation.  The  Fourier  transformation  property  of  a  focussing  lens  is  the  basis  for 
coherent  optical  processing  [Goodman.  Lee].  With  incoherent  radiation,  a  Fourier 
transform  relationship  exists  between  the  intensity  distribution  and  the  spatial  coherence 
of  the  field  at  two  widely  separated  planes.  Once  again,  a  lens  can  be  used  to  shorten 
the  propagation  distance  of  the  field.  Interferometric  optical  processing  architectures 
that  make  use  of  this  coherence  property  of  a  propagating  light  field  are  described  in 
Section  2.4.2. 

2.3.2.3  Convolution  and  Correlation 

Convolution: 

J  J  fi(x,  y)  f2(a-x,  jS-y)  dxdy  =  ^  Fi(u,  v)  F2(u,  v)  ]  (5) 

and  correlation: 

1 1  fi(x,  y)  f2*  (x-a,  y-0)  dxdy  =  ^  Fi(u,  v)  F2*  (u,  v)  ]  (6) 

can  be  performed  by  operating  in  the  spatial  or  spatial  frequency  domain.  In  the  above 
expressions,  [  ]  represents  inverse  Fourier  transformation,  *  denotes  conjugation. 
Implementation  in  the  spatial  frequency  domain  requires  sequential  Fourier 
transformations  and  a  multiplication.  It  is  the  usual  approach  taken  in  coherent  optical 
processing  using  SLMs  as  the  input  device  and  the  spatial  filler.  Incoherent  optical 
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processing  architectures  using  SLRs,  however,  are  not  as  easily  cascaded  to  perform 
sequential  Fourier  transformations.  First  of  all,  the  wavelengths  of  the  input,  readout 
and  output  light  beams  may  not  be  the  same.  More  importantly,  only  positive  real  values 
can  be  represented  by  the  intensity  of  incoherent  light.  Multiple  separate  channels  and 
the  addition  of  a  bias  are  required  to  represent  complex  and  bipolar  values  which  greatly 
complicate  the  implementation  of  sequential  operations. 

For  the  special  case  where  the  input  and  reference  functions,  f^  and  {2  are  both 
positive  and  real,  incoherent  optical  processing  techniques  can  be  used  to  perform 
convolution  and  correlation  operations  directly.  For  example,  the  ouq)ut  of  an  incoherent 
imaging  system  can  be  described  by: 

I(a,  /3)=|  I  |f(x,  y)|^  |h(a-x,  /3-y)l^  dxdy  (7) 

where  I(a,  /3)  is  the  output  image  intensity,  j  h(x,y)  |  ^  is  the  incoherent  point  spread 
function  and  |  f(x,y)  |  ^  is  the  intensity  distribution  of  the  input.  The  spatial  filtering 
of  an  input  image  pattern  can  therefore  be  accomplished  by  designing  the  proper  aperture 
function  F(u,  v)  where  F(u,  v)  =  .?Th(x,y)].  It  is  important  to  note  that  with  such  an 
incoherent  optical  processor,  the  aperture  or  filter  function  F(u,v),  which  can  be  real  or 
complex,  must  be  implemented  with  an  SLM.  The  aperture  mask  in  an  imaging  system 
has  to  operate  on  the  complex  amplitude  of  the  light  field.  A  mask  implemented  with 
an  SLR  would  destroy  all  the  phase  information  of  the  input  field.  The  role  of  the  SLR 
in  such  an  optical  processor  is  therefore  limited  to  the  input  and  output  functions  such 
as  the  removal  of  the  bias  at  the  output  of  the  incoherent  optical  processor  as  described 
in  Section  3.5. 

2.4  Candidate  Optical  Processing  Architectures 

The  choice  of  optical  processing  architectures  is  constrained  by  the  operating 
characteristics  of  SLRs.  First,  the  optical  fields  used  to  write  on  or  read  out  an  SLR  can 
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be  coherent  or  incoherent,  as  long  as  they  are  at  the  proper  wavelengths,  but  the  output 
emission  is  always  incoherent.  The  output  data  are,  therefore,  represented  by  the  beam 
intensity  which  takes  on  only  positive  real  values.  Second,  an  SLR  does  not  simply 
modulate  the  intensity  of  the  input  light  field,  it  also  destroys  all  the  phase  information 
in  the  input  light  field.  Therefore,  SLRs  cannot  be  used  as  direct  replacements  for  SLMs 
in  optical  processors.  Even  with  incoherent  optical  processing  architectures,  the 
destruction  of  the  phase  information  limits  the  functions  that  an  SLR  can  perform.  For 
example,  an  SLR  cannot  be  used  as  ^  aperture  mask  in  an  imaging  system.  The  SLR 
randomizes  the  phase  at  the  aperture  plane  and  no  image  can  form  at  the  output. 


The  active  and  passive  SLRs  also  have  distinct  characteristics  that  affect  the  | 

selection  of  optical  processing  architectures  and  they  must  be  discussed  separately. 

Passive  SLRs:  Being  passive  devices,  they  offer  no  gain.  Moreover,  the  input  and  output 
wavelengths  are  substantially  different.  Together,  they  make  it  almost  impossible  to 
cascade  SLRs  to  perform  sequential  or  iterative  operations.  The  time  constants  for  the 
fluorescence  and  the  stimulated  photoluminescence  are  relative  long  which  limits  the 
cycle  time  of  a  processor  utilizing  passive  SLR.  On  the  other  hand,  the  space-bandwidth 
product  (SBWP)  provided  by  a  passive  SLR  can  be  very  large.  The  spatial  resolution 
of  a  phosphor  based  SLR  can  be  as  high  as  40  Ip/mm.  With  a  25mm  x  25mm  sample, 
the  number  of  resolvable  elements  or  SBWP  is  1000  x  1000.  Optical  processing 
architectures  utilizing  passive  SLRs  must  not  demand  fast  cycle  time  and  should  take  full 
advantage  of  their  large  SBWP. 

Active  SLRs:  Active  SLRs  can  provide  optical  gain  and  the  devices  can  be  easily 
cascaded.  Active  SLRs  can,  therefore,  be  us^  in  an  optical  processing  architectures  that 
perform  sequential  or  iterative  operation.  This  is  important  because  the  SBWP  of  the 
device  is  relatively  small  (<100  x  100).  Image  processing  functions  cannot  be 
performed  on  large  images  in  a  single  pass.  Partitioning  the  image  and  the  processing 
algorithm  is  often  necessary.  Compensating  for  its  small  SBWP  is  the  very  fast  (nsec) 
cycle  time  achievable  with  active  SLRs. 

Passive  SLRs  do  not  possess  strong  nonlinearity.  To  implement  algorithms  such 
as  neural  net  require  strong  nonlinearity,  an  external  means  must  be  used  to  produce  the 
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nonlinearity.  With  an  active  SLR,  on  the  other  hand,  the  desired  nonlinearity  can  be  built 
into  the  SLR. 

These  unique  operating  characteristics  of  active  and  passive  SLRs  were  taken  into 
account  in  the  preliminary  selection  of  optical  processing  architectures  for  SLRs.  The 
architectures  chosen  for  study  were  1)  Scanning  Correlator,  2)  Interferometric  processor, 
3)  OTF  synthesis  Optical  Preprocessor,  4)  Artificial  Neural  Network,  5)  Quadratic 
Processor,  6)  Morphological  Processor  and  7)  Multispectral  Optical  Processor.  The 
implementation  of  these  optical  processing  architectures  with  passive  or  active  SLRs  are 
described  below. 

2.4.1  Scanning  Correlator 

A  scanning  correlator  performs  correlation  in  the  spatial  domain  in  the  form  of: 

fi(a,  /S)  ★  f2(a,  /3)  =  1 1  fi(x,  y)  (x-a,  y-/3)  dxdy  .  (8) 

The  lateral  shifting  of  the  reference  function  f2(x,  y)  encoded  on  an  SLM  is  provided  by 
two  orthogonal  acousto-optic  (AO)  scanners  as  shown  in  Figure  2.4. 1-1,  The  input 
image  from  the  sensor  is  written  onto  an  SLR  and,  at  the  same  time,  the  scanning  image 
of  the  reference  is  used  to  readout  the  SLR.  The  emitted  output  of  the  SLR  corresponds 
to  the  product  of  the  input  and  the  reference  images,  fj(x,y)f2(x-ao,  y-^^.  The  output 
of  the  SLR  is  then  imaged  onto  a  large  area  detector  which  detects  the  total  incident 
optical  power,  effectively  performing  the  spatial  integration.  By  scanning  the  input 
image  in  a  raster  fashion,  the  amount  of  lateral  shift  (a,  /S)  of  the  reference  image  is 
encoded  into  time  by  the  relationship  a  =  Vt/h  -  int(Vt/h)  and  /3  =  int(Vt/h)  where  V 
is  the  velocity  of  the  horizontal  scan,  h  is  the  length  of  the  horizontal  scan  which  is 
determined  by  the  combined  size  of  fj  and  f2  ,  t  is  time  and  int  denotes  the  integer 
function.  The  temporal  output  of  the  detector,  therefore,  represents  the  time  encoded 
spatial  correlation  of  the  input  and  reference  images. 
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Figure  2.4. 1-1.  A  Scanning  Correlator 


34 


The  scanning  correlator  allows  the  image  from  an  optical  sensor  to  be  used  as  the 
input  but  it  requires  an  SLM  for  the  reference  template.  For  many  applications,  it  is  not 
necessary  to  change  the  reference  template  rapidly.  The  use  of  an  SLM  as  the  reference 
does  not  represent  a  significant  disadvantage. 

There  are,  however,  drawbacks  in  the  use  of  a  scanning  correlator.  First  of  all, 
the  correlation  operation  is  performed  via  sequential  steps,  indexing  along  a  and  /S.  The 
operating  speed  is,  therefore,  much  slower  than  a  Fourier  transform  based  correlator  that 
operates  in  the  spatial  frequency  domain.  In  addition,  the  input  and  the  reference 
functions  of  a  scanning  correlator  must  both  be  real  and  positive.  There  is  a  large  body 
of  work  in  optimizing  and  expanding  the  capabilities  of  correlation  filters  to  make  them 
invariant  to  the  size,  orientation  and  aspect  of  the  target,  and  more  tolerant  of  variation 
in  lighting  conditions  and  partial  obscuration.  These  filters,  unfortunately,  require  the 
reference  function  to  be  complex.  A  scanning  correlator  cannot  take  advantage  of  these 
recent  advances  in  correlation  filter  design. 

2.4.2  Interferometric  Processor 

From  coherence  theory,  it  is  well  known  that  a  Fourier  transform  relationship 
exist  between  the  intensity  distribution  and  the  coherence  of  the  radiation  in  the  far  field 
[Bom].  That  is: 

ti{u,  V)  =  I  |l(a,  jg)  g-*'2T(«a^vft/X 

where  n(u,  v)  s  the  complex  degree  of  spatial  coherence  (CDSC),  I(a,  0)  is  the  intensity 
distribution  of  the  input  function  and  £  is  the  field  of  view  of  the  sensor.  Using  a 
shearing  interferometer  [Aleksoff,  TaiJ  to  measure  the  CDSC  at  a  range  of  spatial 
frequencies  simultaneously,  the  Fourier  transform  of  the  intensity  of  the  target  scene  can 
be  obtained  at  the  speed  of  light  without  requiring  an  intervening  spatial  light  modulator. 
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The  output  intensity  of  a  shearing  interferometer,  I,,  ,  is  given  by: 


Io(“.  V)  = 


4 


(JC.  }’){1  cos[2ir(ux+yy)]}  dxdy 


/,(x,  y)  dxdy^^  |  /,  (x,  y)  cos[2-r(itt  +  yy)]  dxdy 


(10) 


=  Bias  +  Re{^  [/,(x,  y)]} 


To  obtain  the  imaginary  part  of  the  Fourier  transform,  a  r/2  phase  shift  is  inserted  in  one 
of  the  sheared  beams  to  produce: 


Io(“>  V)  = 


1 1  iM,  y)  dxdy*^  I  /,(3C,  y)  sin[2T(ia  +  vy)]  dxdy 


(11) 


=  Bias  +  Im{^  [7,(x,  y)]} 


A  rotation  shearing  interferometer  such  as  a  modified  Koster  interferometer 
[Aleksoff]  as  illustrated  in  Figure  2.4.2-1  can  provide  the  two-dimensional  transform  of 
an  incoherent  input  pattern.  However,  the  interferometer  can  only  operate  with  narrow 
band  radiation  and  the  image  of  the  target  scene  cannot  be  used  directly  as  the  input.  A 
grating  interferometer  is  achromatic  [Tai]  and  can  operate  directly  on  the  incoherent 
radiation  from  the  target  scene.  The  grating  interferometer,  however,  is  a  one¬ 
dimensional  device.  For  a  two-dimensional  input,  the  interferometer  output  corresponds 
to  a  radial  line  in  the  Fourier  spectrum.  A  two-dimensional  transform  of  the  input  can 
be  obtained  by  rotating  the  interferometer  about  its  optical  axis. 


Since  the  bipolar  output  is  on  a  bias,  the  signal  is  often  overwhelmed,  particularly 
if  the  size  of  the  image  support,  E,  is  large.  For  the  processor  to  be  viable,  a  means  to 
reduce  the  bias  is  necessary.  The  fact  that  the  real  and  imaginary  parts  of  the  Fourier 
transform  reside  at  the  outputs  of  two  separate  channels  also  makes  cascading  the 
processor  more  difficult. 


< 
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Figure  2. 4. 2-1.  A  Rotation-Shearing  Interferometer 


2.4.3  OTF  Synthesis  Optical  Preprocessor 

As  described  in  Section  1.4.2. 3,  incoherent  optical  processing  techniques  can  be 
used  to  perform  convolution  and  correlation  operations  for  the  special  case  where  the 
input  and  reference  functions  are  both  positive  and  real.  Specifically,  the  output  of  an 
incoherent  imaging  system  can  be  described  by: 

I(a,  /3)=|  I  lf(x.  y)|^  |h(a-x,  |8-y)l^  dxdy  (^2) 

where  I(a,  0)  is  the  output  image  intensity,  |  h(x,y)  |  ^  is  the  incoherent  point  spread 
function  and  (  f(x,y)  |  ^  is  the  intensity  distribution  of  the  input.  The  Optical  Transfer 
Function  (OTF)  of  the  imaging  system  is  defined  as  the  normalized  Fourier  transform 
of  I  h(x,y)  I  ^  .  That  is,  [Rhodes]: 


OTF(u,  V)  =  .  (13) 

1 1  |h(x,  y)|2  dxdy 

Spatial  filtering  of  an  incoherent  input  image  can  be  accomplished  by  designing  the 
proper  aperture  function  F(u,  v)  where  F(u,  v)  =  .^(x,y)].  We  note  that  F(u,v)  can 
be  complex.  Complex  operations  such  as  matching  filtering  can  therefore  be  performed. 

F(u,v)  must  be  implemented  with  an  SLM.  The  filter  function  may  be  complex 
as  we  have  just  indicated.  More  importantly,  the  coherence  and  the  phase  of  the  input 
field  must  be  preserved  by  the  spatial  filter  which  is  not  the  case  with  SLRs  where  the 
input  and  output  photons  are  different. 

2.4.4  Artificial  Neural  Networks 

Artificial  neural  networks  consist  of  nodes  (or  artificial  neurons)  which  are 
connected  together  [Lippman].  Figure  2,4.4-l  shows  a  node  with  inputs  x  and  output  y. 
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Figure  2.4  4-1.  Basic  Processing  Element  of  an  Artificial  Neural  Network 


The  node  performs  the  function  of  weighting  each  input  with  a  corresponding  weight  w, 
summing  these  weighted  inputs,  subtracting  a  threshold  O,  and  passing  the  result  through 
a  nonlinearity  f( )  to  produce  the  output  y.  Sample  nonlinear  functions  f  are  also  shown 
in  Fig.  2.4,4-l.  Typically,  nodes  are  grouped  to  form  layers  and  the  layers  are 
massively  interconnected.  Figure  2.4.4-2  shows  a  three-layer  perceptron  artificial  neural 
network.  The  N  inputs  x  are  applied  to  the  first  layer  of  N  nodes.  The  N  outputs  x’  of 
the  first  layer  serve  as  inputs  to  second  layer  and  so  on  to  the  final  N  outputs  y.  Layers 
which  neither  receive  the  initial  input  nor  produce  the  final  output  are  called  hidden 
layers.  Many  other  connections  are  possible  including  feedback  connections,  but  a  single 
network  seldom  has  more  than  three  layers.  Entire  networks  may  sometimes  be 
cascaded,  however. 

The  computations  found  in  artificial  neural  networks  are  multiplication,  addition, 
and  nonlinearity.  Artificial  neural  networks  also  require  massive  data  broadcasting  and 
reception.  Linear,  passive  optical  processing  techniques  can  provide  the  multiplication 
and  addition.  Optical  methods  may  also  be  used  to  provide  the  massive  interconnections. 
Nonlinear  optics  or  opto-electronic  spatial  light  rebroadcasters  can  provide  the  nonlinear 
operations.  A  schematic  of  a  possible  opto-electronic  node  is  shown  in  Fig.  2.4.4-3. 
The  weighted  inputs  are  summed  as  they  strike  detectors.  Incoherent  optical  processing 
is  assumed  and  two  channels  are  shown  to  handle  positive  and  negative  weight  values. 
The  op-amp  performs  subtraction  of  the  positive  and  negative  channels  and  applies  the 
nonlinearity.  The  output  is  rebroadcast  to  the  next  layer  by  a  laser  diode. 
Implementations  with  light  emitting  diodes  are  equally  possible. 

Five  optical  architectures  for  artificial  neural  networks  were  studied.  The  first 
three  assume  fixed  weight  values  and  use  holographic,  cylindrical,  and  lenslet  array 
optics  respectively  to  perform  the  interconnections.  The  fourth  architecture  assumes 
fixed  weights  and  uses  cylindrical  optics,  but  uses  a  phosphor  type  spatial  light 
rebroadcaster.  The  fifth  architecture  includes  optical  training  or  computation  of  the 
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Figure  2. 4.4-2.  A  Three-Layer  Perceptron  Neural  Network 
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Figure  2. 4.4-3.  An  Opto-electronic  Node  in  a  Neural  Network 
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weight  values.  All  of  the  architectures  are  shown  for  a  single  layer  artificial  neural 
network.  They  would  be  cascaded  for  multiple  layer  networks. 

2.4.4. 1  Holographic  Architecture 

Figure  2. 4. 4-4  shows  the  holographic  artificial  neural  network.  At  each  node, 
holographic  light  redistribution  elements  are  located  which  consist  of  superimposed  or 
spatially  multiplexed  gratings.  These  elements  take  the  output  of  a  node,  break  it  into 
multiple  parts,  weight  each  part,  and  send  each  part  toward  a  specific  node  in  the  next 
layer.  To  do  this,  each  grating  has  a  diffraction  angle  appropriate  to  diffract  the  light 
to  a  specific  node  and  a  diffraction  efficiency  corresponding  to  the  weight  required  for 
that  input  to  the  node.  The  nodes  would  be  opto-electronic  spatial  light  rebroadcasters, 
for  example  of  the  type  shown  in  Fig.  2. 4.4-3.  Although  shown  as  one-dimensional  in 
the  figure,  the  processor  could  operate  in  two  dimensions. 

In  addition  to  the  spatial  light  rebroadcasters,  the  critical  element  of  this 
architecture  is  the  fabrication  of  the  holographic  elements.  Figure  2.4.4-5  shows  one 
method  for  fabrication  in  which  an  SLM  would  be  used  to  program  the  recording  beams 
to  write  individual  gratings  on  a  holographic  recording  media.  Binary  optics  fabrication 
techniques  could  also  be  used.  It  will  be  difficult  to  maintain  sufficient  accuracy  in  the 
weights  (diffraction  efficiencies).  Other  studies  performed  by  ERIM  have  shown  that  3% 
accuracy  (5  bits)  is  needed  at  a  minimum  for  successful  neural  net  operation 
[Cederquist].  This  would  be  difficult  for  the  spatially  multiplexed  gratings,  let  alone  for 
the  superimposed  grating  approach. 

2.4.4.2  Cylindrical  Optics  Architecture 

Figure  2.4. 4-6  shows  a  cylindrical  optics  based  artificial  neural  network.  The 
input  is  a  linear  array  of  laser  diodes  or  light  emitting  diodes  each  of  which  is  collimated 
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Figure  2  4  4-4.  A  Holographic  Neural  Network 
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Figure  2. 4.4-5.  Fabncat'on  of  Holographic  Grating  for  Neural  Network 
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Figure  2.4  4-6  Cylindrical  Optics-Based  Artificial  Neural  Network 
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by  the  first  cylindrical  optic.  The  resulting  light  passes  through  an  intensity  (or  gray 
scale)  mask  where  it  is  multiplied  by  the  weights.  A  second  cylindrical  optic  oriented 
at  90  degrees  to  the  first  focuses  this  light  onto  a  linear  array  of  opto-electronic  spatial 
light  rebroadcasting  devices.  At  their  detectors  summation  is  performed,  nonlinearity  is 
performed  electronically,  and  the  output  is  optically  rebroadcast  to  the  next  layer. 

The  critical  elements  of  this  architecture  are  the  performance  of  the  cylindrical 
optics  and  the  fabrication  of  the  two-dimensional  weight  mask.  Fabrication  of  a  spatially 
multiplexed  weight  mask  to  1  %  accuracy  can  be  achieved  with  table  look-up  linearization 
[Cederquist  and  Lee].  This  architecture  is  known  in  the  literature  and  some  experimental 
results  for  a  matrix- vector  multiplier  have  also  already  been  published,  so  the  risk  of  the 
cylindrical  optics  was  judged  to  be  low.  Therefore,  this  architecture  was  chosen  over 
the  other  approaches  for  in-depth  analysis, 

2.4.4.3  Lenslet  Array  Architecture 

The  third  architecture  studied  is  shown  in  Fig.  2.4.4-7,  It  is  similar  to  the  second 
architecture  in  that  it  uses  a  spatially  multiplexed  weight  mask,  but  it  uses  lenslets  to 
image  the  input  onto  the  mask  and  lenslets  to  collect  the  light  onto  the  detectors.  The 
critical  elements  of  this  architecture  are  the  performances  of  the  two  lenslet  arrays.  Each 
element  of  the  first  array  must  demagnify  the  input  and  image  it  to  a  specific  location  on 
the  mask.  This  means  the  input  FOV  of  the  lenses  is  large  and  the  lenses  will  have 
different  vignetting  over  their  FOV  depending  on  their  location  in  the  array.  This 
variation  would  have  to  be  computed  and  compensated  in  the  weight  mask.  Light  leaving 
the  mask  does  not  in  general  propagate  along  the  optical  axis,  but  at  an  angle  to  it  which 
becomes  greater  toward  the  edges  of  the  mask.  This  requires  lenslets  of  the  second  array 
to  be  capable  of  capturing  and  focusing  light  arriving  at  large  angles  to  their  optical  axes. 
On  balance,  this  architecture  seemed  to  have  no  advantage  of  size  and  probable 
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Figure  2.4  4-7.  Lenslet  Array  Architecture  for  Artificial  Neural  Network 
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disadvantages  in  accuracy  and  complexity  (use  of  lenslet  arrays)  over  the  second 
architecture  and  so  was  not  analyzed  further, 

2.4.4.4  Phosphor  Based  Architecture 

The  fourth  architecture  studied  is  shown  in  Fig.  2. 4. 4-8.  It  uses  the  cylindrical 
optics  architecture,  but  substitutes  a  phosphor  type  of  spatial  light  rebroadcaster  for  the 
opto-electronic  type.  This  is  done  by  having  two  inputs,  one  at  a  write  wavelength  for 
positive  weight  values  and  one  at  an  erase  wavelength  for  negative  weight  values.  Each 
is  multiplied  by  the  weights  using  a  mask,  but  the  subtraction  is  done  optically  by  writing 
and  erasing  the  phosphor  rather  than  electronically  in  an  opto-electronic  spatial  light 
rebroadcaster.  The  critical  element  of  this  architecture  is  the  requirement  of  the  spatial 
light  rebroadcaster  to  perform  the  nonlinear  operation  required  by  artificial  neural 
networks.  As  discussed  in  Section  3.S,  accurate  subtraction  followed  by  nonlinearity  is 
not  a  natural  operation  of  phosphors.  In  addition,  this  architecture  is  not  optically 
cascadable  since  the  wavelength  of  the  rebroadcast  is  not  suitable  for  either  the  write  or 
erase  inputs.  For  these  reasons,  this  architecture  was  not  analyzed  in  depth. 

2.4.4.5  Adaptive  Weight  Architecture 

The  fifth  architecture  studied  is  shown  in  Figure  2.4.4-9.  Unlike  the  others,  it 
does  not  use  fixed  weights,  but  electronically  computes  any  necessary  changes  to  the 
weights  (such  as  occurs  during  artificial  neural  network  training)  and  optically  updates 
the  weight  mask.  The  architecture  uses  cylindrical  optics  with  the  weight  mask  written 
on  a  phosphor  spatial  light  rebroadcaster.  Since  this  weight  mask  will  be  partially  erased 
each  time  it  is  read  out,  it  is  refreshed  (rewritten)  every  10  cycles  by  a  weight  mask  in 
digital  electronic  memory  and  displayed  on  a  CRT.  The  output  of  the  artificial  neural 
network  processor  is  not  only  rebroadcast  to  the  next  processor,  but  is  also  input  to  a 
digital  electronic  processor.  In  the  digital  electronic  processor,  this  input  is  used  to 
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yjJ^rase  input 
A, 


•  Use  write,  erase  wavelengths  in  input  and  subtract  at  SLR 

•  Requires  phosphor  SLR  with  nonlinearity 

•  Not  cascadable 


Figure  2. 4. 4-8.  Phosphor  Passive  SLR-Based  Architecture  for  Artificial  Neural  Network 
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Figure  2  4.4-9.  Adaptive  Weight  Architecture  for  Artificial  Neural  Network 


compute  changes  to  the  weights  (training  the  artificial  neural  network).  Any  changes  to 
the  weights  are  fed  to  scanned  laser  diodes  or  light  emitting  diodes  which  write  or  erase 
the  weight  mask  stored  on  the  phosphor  spatial  light  rebroadcaster.  For  a  multi-layer 
perceptron  neural  network,  training  would  traditionally  be  done  by  the  back-propagation 
algorithm  and  the  computational  load  on  the  electronic  processor  would  limit 
performance.  Neither  would  an  optical  processor  have  sufficient  flexibility  to  compute 
this  algorithm.  However,  for  a  Kohonen  self-organizing  neural  network,  the  weight 
modification  algorithm  is  much  simpler  and  only  small  regions  of  the  weight  mask  need 
to  be  modified  at  each  cycle.  For  this  case,  the  load  on  the  electronic  processor  would 
not  be  the  limiting  factor.  Instead  the  overall  computation  rate  would  be  determined  by 
the  space-bandwidth  product  of  the  CRT  and  the  rate  at  which  weight  changes  could  be 
optically  made.  Therefore,  if  real-time  artificial  neural  network  training  is  required, 
then,  for  networks  like  the  Kohonen,  this  architecture  is  a  candidate. 

2.4.5  Quadratic  Processor 

The  optical  architecture  that  we  will  discuss  in  this  section  will  affect  the 
operation  of  quadratic  systems.  Quadratic  systems  are  extremely  important  in  a  number 
of  applications  [Rugh].  We  will  focus  on  automatic  target  detection  operations  which  is 
the  primary  goal  of  these  processors.  The  general  input/output  relationship  is  given  as: 

=  I  q{r-r^,  r-r^  dr^  dr^ 

where  fir)  is  the  input  and  q(r^,  r2  )  is  the  kernel  function.  For  sampled  data,  the 
quadratic  processor  can  be  rewritten  in  vector/matrix  notation  as: 

g  ’fQf 

where /is  the  input  signal  vector,  g  is  the  output  vector,  and  Q  is  the  kernel  matrix  and 
T  denotes  vector  transpose. 
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The  quadratic  operation  is  one  of  the  most  important  operations  in  target 
detection.  The  target  detection  problem  can  be  seen  as  a  binary  hypothesis  test  where 
the  two  hypotheses  are: 

Hj!  clutter  with  distribution  Pi(/)=N(/ii ,  Ej) 

H2;  target  plus  clutter  with  distribution  P2(/)=N(^2  »  ^2) 

where  N  denotes  multivariate  Gaussian  distribution,  with  i^i  ,  and  fi2  are  the  mean 
vectors  and  Ej  ,  and  E2  as  the  covariance  matrices  of  the  two  distributions. 

The  optimal  test  to  differentiate  between  these  two  hypotheses  is  the  likelihood 
ratio  test  (LRT)  [VanTrees].  In  this  test,  the  ratios  of  the  two  probability  distribution 
functions,  A(/)  =  Pi(^/Po(/)  are  calculated.  This  will  produce  an  operation  on  the 
incoming  data  which  is  then  compared  to  a  threshold.  We  will  assume  the  input  signal 
vector  f  to  be  provided  by  an  electro-optical  sensor.  In  this  instance,  /can  be  composed 
of  multispectral  and/or  spatial  data.  When  /is  spatial  data  from  the  local  neighborhood 
of  the  pixel  under  test,  the  LRT  effectively  compares  the  local  spatial  texture  for  target 
detection.  This  has  been  shown  to  provide  significant  detection  advantage  of  pure  energy 
detection  [STAR  report].  For  an  input  data  vector /  the  LRT  becomes: 

The  first  term  is  the  quadratic  operation  where  the  kernel  is  represented  by  Q=(Ef^  - 
E2'^).  These  two  covariance  matrices  can  be  computed  off-line  via  training  data  of  both 
the  clutter  and  target.  The  second  term  is  a  simple  linear  processor  with  impulse 
response  h=E/Vo "  Vz-  This  term  can  be  eliminated  entirely  when  "demeaned"  data 
are  used.  We  will  consider  this  case  in  our  subsequent  discussions. 

An  optical  implementation  of  the  quadratic  operation  is  shown  in  Figure  2.4. 5-1. 
This  architecture  has  some  significant  advantages  when  compared  to  its  electronic 
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Figure  2. 4. 5-1.  Optical  Quadratic  Processor 


counterparts.  The  first  and  foremost  is  that  this  processor  computes  the  LRT  for  each 
pixel  in  the  scene  simultaneously.  In  electronic  implementations,  a  window  is  translated 
around  the  image  and  the  LRT  is  computed  serially.  The  SLR  implementation  allows 
the  system  to  operate  directly  on  the  optical  intensity  at  the  focal  plane  of  the  sensor. 
No  arbitrary  optical/electronic-electronic/optical  transduction  (i.e.,  a  detector  array 
followed  by  a  spatial  light  modulator)  is  required. 

There  are  two  specific  technologies  used  in  the  quadratic  processor  architecture, 
the  spatial  light  rebroacaster,  and  lenslet  arrays.  The  spatial  light  rebroadcaster  allows 
real  time  data  insertion,  filtering  and  thresholding  whereas  the  lenslet  arrays  provide  the 
means  for  local  neighborhood  isolation,  pixel  replication  and  local  spatial  integration. 
The  first  SLR  in  the  architecture  detects  and  rebroadcasts  the  sensor  data  into  the  optical 
processor.  The  first  lenslet  array  selects  (through  reimaging)  a  local  3x3  neighborhood 
around  each  pixel  in  the  image.  This  requires  the  same  number  of  lenslets  as  the  number 
of  pixels  in  the  input  image.  The  second  lenslet  array  replicates  each  neighborhood  3x3 
times  (also  through  reimaging).  The  replicated  neighborhood  are  incident  on  the  filtering 
SLR  where  the  matrix  Q  is  stored.  The  third  lenslet  array  integrates  each  3x3  segment 
and  images  the  result  onto  the  third  SLR.  This  SLR  also  has  the  selected  local 
neighborhood  data  as  a  readout  beam  incident  on  it  via  a  network  of  two  beam  splitters, 
a  lenslet  array  for  neighborhood  selection  and  a  one-to-one  imaging  system.  The  SLR 
then  produces  the  multiplication  of  the  two  data  sets.  A  final  lenslet  array  integrates  the 
local  3x3  output  of  the  SLR  and  images  the  result  to  a  final  active  SLR  which  provides 
the  thresholding  operation.  The  output  "image”  of  this  system  is  a  binary  mapping 
whose  pixels  are  either  detection  of  target  (on)  or  clutter  (ofO-  Refer  to  section  3.3  for 
specific  operational  details. 

The  initial  assessment  of  benefit  for  the  optical  processor  can  be  calculated  via 
its  throughput.  We  will  make  this  calculation  based  on  the  number  of  operations  that  are 
applied  to  any  specific  input  pixel.  The  key  to  this  calculation  is  the  realization  that  each 
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input  pixel  is  replicated  81  times  (local  neighborhood  selection  and  neighborhood 
replication).  Note  that  each  pixel  is  contained  in  the  neighborhoods  of  all  9  of  its 
neighbors.  This  is  shown  in  Figure  2.4. 5-2.  The  filtering  SLR  then  performs  81 
multiplies  on  a  particular  input  pixel.  The  lenslet  array  following  the  filtering  SLR 
produces  8  additions  (replicated/ filtered  pixel  being  added  to  each  element  in  its 
neighborhood)  for  each  replicated  neighborhood  (9  times).  This  produces  72  additions 
on  each  input  pixel.  Multiplication  by  the  neighborhood  data  at  the  third  SLR  provides 
another  9  multiplications  per  pixel.  Finally,  the  last  lenslet  array  sums  over  each  local 
neighborhood  (8  additions).  The  thresholding  operation  will  be  ignored  in  this 
calculation.  This  produces  a  total  of  170  operations  on  each  input  pixel.  This  is 
summarized  as: 


Multiplication  by  mask: 

Lens  summation  (8  adds  x  9  real.) 
Multiplication  by  neighborhood 
Final  summation 


81  multiplications 
72  additions 
09  multiplications 
08  additions 


Total  170  ops/input  pixel 

We  will  assume  an  input  image  size  of  500  x  500  pixels  (2.5  x  10^  pixels).  For  an  SLR 
response  time  of  30  msec  (30  frames/sec)  the  processor  architecture  effectively  operates 
at  a  throughput  rate  of  1.4  Gops/sec.  Conversely,  for  an  SLR  response  time  of  1  msec, 
the  processor  throughput  rate  is  42.5  Gops/sec.  Clearly,  this  is  significantly  higher  than 
any  low  power  electronic  system  can  achieve.  Therefore,  we  advise  that  the  quadratic 
architecture  introduced  in  this  section  be  further  studied. 
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Figure  2.4. 5-2,  3x3  Neighborhood  Operation 
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2.4.6  Morphological  Processor 


Morphological  image  processing  has  been  applied  successfully  in  a  variety  of 
applications  including  automatic  target  recognition  and  classification  [Sternberg, 
Maragos,  Crimmins].  Morphological  processing  is  based  on  a  series  of  local  operations 
and  neighborhood  transformations  which  are  performed  identically  on  all  image  pixels. 
This  translation-invariant  property  makes  morphological  processing  suitable  for  optical 
implementation  and  allows  the  parallelism  to  take  full  advantage  of  an  optical  processor. 

In  this  section,  morphological  image  processing  is  first  reviewed  and  the 
implementations  of  its  basic  operations  with  an  incoherent  optical  processor  are 
described.  Processing  algorithms  for  ATC  applications  based  on  a  series  of  these 
elementary  operations  and  transformations  are  then  discussed  and  the  system  architectures 
for  a  hybrid  optical  morphological  image  processor  are  examined. 

2.4.6. 1  Image  Algebra 

It  is  well  known  in  image  algebra  that  there  are  two  fundamental  local  operations 
and  one  neighborhood  transformation  with  which  most  other  operations  or  transforms  can 
be  implemented.  The  two  fundamental  local  operations  are:  [Serra,  Huang]: 

Complement  of  an  image: 

A  =  {(Jc,y)|(x,y)  e  Wand  (x,y)  €  A} 
where  W  is  the  image  space  containing  all  image  pixels. 
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Union  of  two  images  A  and  R: 


A  [j  R  ^  {(j:,y)|(x,y)  G  A  or  (xj)  G  R} 
The  fundamental  neighborhood  transformation  is 

Dilation  of  image  A  by  Ri 


(15) 


{(x^ *yr)  ^  r)  ^ 


A  ©  =  1 


0 


(A^f3  )  and  {R^0  ) 

otherwise 


(16) 

where  0  denotes  the  null  image  set.  In  other  words,  dilation  is  the  union  of  the 
translation  of  A  by  the  elements  in  R.  That  is,  if  we  let  p  be  a  pixel  in  W  and  denote 
the  shifting  of  the  origin  of  A  to  p  by: 

A(p)  =  {a+p  I  aGA} 
then  dilation  can  also  be  expressed  as: 

A®  R  =  (J  A(ri)  (17) 

TjGR 

where  A(rj,  r2,....rj^)  are  the  translated  images  of  A  by  the  pixels  in  /?  =  {r^, 

Based  on  these  three  fundamental  operations  and  transformations,  other  operations  and 
transformations  can  be  implemented.  For  example: 
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Erosion: 


AQ  R  =  (a®R) 


(18) 


where  Tt  denotes  the  symmetric  set  of  R,  that  is,  the  rotation  of  R  by  180°: 


Difference: 


^  =  (A  U 

Intersection: 


(19) 


^  n  ^  =  (A  u  /?) 


(20) 


Symmetric  difference: 


A  A  R  =  B  R)U  {A  \J  R) 


(21) 


Opening: 


=  (A  ®  #?)  ®  ^  ®  ®  ^ 


Closing: 


==^— 

A^  =  {A  B  R)  ®  R  =  {A  ®  R)  Q  R 

The  operations  and  transformations  are  illustrated  in  Figure  2. 4. 6-1  for  the  case  R  =  R. 
Other  more  complex  transformations  include: 


Hit  or  Miss  transform: 

A  ®  /?  =  (A  ©  i?i)  -  (A  ©  /?2)  =  OK  ©  R])  n  (^  Q  Rj) 
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Figure  2.4.6-]  Morphological  Operations  and  Transformations  that  can  be 
Implemented  with  Complement,  Union  and  Dilation 
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Thinning: 


A  O  R  =  A  -  (A®/?)  (25) 

Thickening: 

AOR^A[j{A®R)  (26) 

and  Skeletonization: 

SK(A)  =  U  {04  ©  r«)p3}  (27) 

r  >0 

where  tB  is  an  open  disk  of  radius  r  and  p  is  a  closed  disk  of  the  size  of  a  single  pixel, 
and  (Op  denotes  the  opening  of  (•)  by  p. 

In  image  processing,  A  is  generally  the  input  image  and  R  is  referred  to  as  the 
structure  element  which  operates  on  A.  .R  can  be  of  any  shape.  Some  commonly  used 
shapes  are  circle  (disk),  square,  line  and  rhombus. 

2.4.6.2  Optical  Implementation  of  Elementary  Operations  and 
Transformations 

As  described  in  Section  1.0,  there  are  two  elementary  local  operations  and  one 
neighborhood  transformation  with  which  other  morphological  operations  and 
transformations  can  be  implemented,  namely,  complement,  union  and  dilation.  In  this 
section,  the  optical  implementations  of  these  elementary  functions  are  described.  The 
system  architecture  of  an  optical  morphological  processor  is  discussed  later  in 
Section  5.0. 

Complement 

Complement  or  negation  requires  a  device  with  an  input-output  intensity  transfer 
characteristic  that  has  a  negative  slope.  Many  spatial  light  modulators  can  be  made  to 


produce  positive  or  negative  outputs  by  varying  the  bias  intensity  or  voltage  levels.  The 
SLR  is  assumed  to  also  have  the  capability  to  be  switched  to  produce  a  positive  or 
negative  output. 

Union 

Union  is  equivalent  to  a  logical  OR  operation.  It  can  be  implemented  with 
summation  and  thresholding  operations.  That  is: 

A  U  ^  =  {(A:,)')l(A:,y)  e  [k^(x,y)  +  ^j|f(r,y)]  >  1}  (28) 

where  A  and  R  are  binary  images  with  values  of  0  and  1,  and  kg  are  the  indicator 
functions  associated  with  compact  sets  A  and  R.  The  summation  of  intensity  is  a  natural 
function  of  a  square  law  detector  such  as  a  SLR.  Two  images  incident  on  the  SLR 
simultaneously  will  be  summed.  With  the  storage  capability  of  the  SLR,  the  summation 
can  also  be  performed  on  images  which  impinge  on  the  SLR  sequentially.  Thresholding 
can  be  implemented  with  an  active  SLR  by  the  appropriate  design  of  the  electronics 
between  the  detectors  and  the  emitters  of  the  device. 

Dilation 

Dilation  is  the  union  of  the  translations  of  A  by  the  elements  in  R.  It  can  be 
expressed  in  the  form  of  a  convolution  operation; 

A  ®  /f  =  o[k^{x,y)  *  Arj^(x,y)]  (29) 

where  *  denotes  the  convolution  operation  and  a  indicates  the  support  of  convolution 
product.  The  convolution  can  be  performed  optically  and  the  support  can  be  obtained 
by  simple  thresholding. 

With  an  incoherent  imaging  system,  the  output  image  intensity  for  an  input, 
is  given  by  k^(x,y)  *  lh(x,y)|^  where  lh(x,y)p  is  the  point  spread  function  of 
the  imaging  system.  It  is  related  to  the  pupil  function,  Pfu.v).  via  h(x,y)  =  .F|P(u,v)] 
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where  ]  denotes  Fourier  transformation.  Dilation  can,  therefore,  be  performed  by 
choosing  a  pupil  function  P(u,v)  whose  point  spread  function  matches  the  desired 
structure  element  R.  When  the  structure  element  is  large,  the  pupil  function  becomes 
small,  severely  affecting  the  optical  efficiency  of  the  optical  processor.  A  simple 
means  to  improve  the  optical  throughput  through  the  lens  aperture  is  to  replicate  the  pupil 
function  as  illustrated  in  Figure  2. 4. 6-2.  The  spatial  separation  of  the  pupil  functions  has 
to  be  large  enough  such  that  the  resulting  spurious  fringe  pattern  cannot  be  resolved  by 
the  out  detector  array  or  SLR. 

The  point  spread  function  is  always  real  and  positive.  Performing  the  convolution 
with  an  incoherent  imaging  system  will  restrict  the  type  of  structure  elements  that  can  be 
realized  directly.  With  dilation,  the  structure  element  is  defined  only  by  points  which 
belongs  to  R.  (In  hit  or  miss  transform  which  will  be  described  in  the  following  section, 

the  structure  element  is  also  defined  by  points  that  belong  to  the  background  or  /? .)  kjf 
is,  therefore,  real  and  positive  and  the  convolution  can  be  realized  with  an  incoherent 
imaging  system. 

To  summarize,  A  ©  /?  can  be  implemented  by  thresholding  the  output  image 
obtained  with  an  incoherent  imaging  system  having  a  pupil  function,  P(u,v),  where: 

J)  =  [P(w.v)l  P  • 

A  detailed  description  of  the  pupil  function  design  for  morphological  processing  will  be 
presented  later  in  Section  3.4, 

Hit  or  Miss  Transformation,  Thinning  and  Thickening 

In  Hit  or  Miss  transform,  A  ©  /?,  the  structure  element  is  composed  of  two 
components  /?j  and  R2  where  R^  is  defined  by  points  that  belong  to  A  (shape  of 

foreground)  and  R2  is  defined  by  points  that  belong  to  A  (shape  of  background).  The 
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Pupil  function  that  produces  disk-like  structure  element 


Figure  2.4. 6-2.  Replication  of  Pupil  Function  to  Increase  Optical  Throughput 
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condition  of  the  transform  is  that  A  matches  /fj  and  A  matches  Jl2-  A  simple  example 
is  shown  in  Figure  2.4.6-3.  which  performs  an  asymmetric  form  of  edge  filtering  where 
only  the  left  vertical  edge  is  enhanced. 

Closely  related  to  hit  or  miss  transform  are  thinning  and  thickening.  Thinning  is 
obtained  by  subtracting  X®  R  from  A.  Thickening  is  obtained  by  taking  union  of  A  with 
A®  R. 

With  the  structure  element  composing  of  two  components,  the  hit  of  miss 
transform  (Eq.  24)  can  be  implemented  optically  using  two  pupil  masks  to  produce  if  j 

and  R2.  The  two  parts  of  the  hit  or  miss  transform,  (X  ©  /fj)  and  (X  ©  if 2),  can  be 
performed  with  parallel  imaging  optics  and  then  summed  and  thresholded  to  obtain  the 
intersection.  Alternatively,  the  operations  can  be  implemented  sequentially  using  the 
storage  capability  of  the  SLR. 

Some  commonly  used  structure  elements  are  given  in  Figure  2.4,6-4.  Notice  that 
the  centers  of  if  j  and  R2  are  often  off-set  from  each  other.  With  an  incoherent  optical 
system,  however,  the  point  spread  function  convolving  with  the  input  image  is  always 
centered  at  the  optical  axis.  A  means  must,  therefore,  be  available  to  shift  one  of  the  two 
output  images  before  they  are  summed  and  thresholded.  The  amount  of  shift  required 
is  quite  small.  The  shift  may  be  obtained  by  adding  a  grating  structure  to  the  pupil 
function  if  the  programmable  pupil  mask  (SLM)  provides  enough  space-bandwidth 
product.  Alternatively,  it  can  be  achieved  with  electro-mechanical  means  by  tilting  a 
mirror  or  a  beamsplitter  with  a  piezoelectric  driver.  The  shift  can  also  be  performed 
electro-optically  by  adding  a  wedge  of  electro-optics  material  at  the  pupil  plane.  The 
application  of  voltage  changes  the  amount  of  linear  phase  retardation  and  shifts  the  output 
image.  It  may  also  be  possible  to  shift  the  image  stored  in  an  active  SLR  electronically 
by  using  a  charge  couple  device  structure  to  fabricate  the  SLR. 
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Struciuring  elements 


For  X 

For  r  ^={p#} 


•  points  of  the  structuring  element  which  must  belong  to  X 
©  ptoints  of  the  structuring  element  which  must  belong  to  X 

i  location  of  the  origin  associated  with  the  structuring  element. 


Figure  2.4. 6*3,  Example  of  Hit  or  Miss  Transformation  for  Spatial  Filtering 
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Figure  2A.6-4.  Some  Commonly  Used  Structure  Hements 
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2.4.6.3  Extension  to  Gray  Scale  Images 

So  far,  we  have  limited  ourselves  to  binary  input  images.  In  this  section, 
approaches  to  allow  the  optical  morphological  processor  to  handle  gray  scale  images  are 
described. 

Gray  scale  morphological  operations  are  very  computation  intensive.  A  gray  scale 
ojjening  for  example,  requires  much  more  computation  than  a  thresholding  operation 
followed  by  a  binary  opening.  For  high  speed  processing,  gray  scale  2-dimensional 
images  are  typically  decomposed  into  2-dimensional  binary  images  which  are  then 
processed  efficiently  by  binary  morphological  operations  and  transformation. 

Two  approaches  to  decompose  a  gray  scale  image  into  binary  images  will  be 
described.  The  most  powerful  approach  utilizes  a  concept  called  umbra  with  which  gray 
scale  images  can  be  processed  in  3-space. 

2.4.6.3.1  Umbra 

A  gray  scale  image  can  be  considered  to  be  a  binary  three-  dimensional  (i,x,y) 
image  and  morphological  operations  and  transformations  can  be  performed  by  breaking 
the  gray  scale  image  into  a  series  of  binary  two-dimensional  images.  Let  A  be  a  set  of 
points  in  2-space  and  G  is  a  function  which  assigns  a  gray  scale  value  y  to  each  point 
(x,y)  in  G{A^)  is  then  a  gray  scale  image  having  a  binary  representation  in  3-space, 
p^  is  an  element  in  A^  if  p^  is  an  element  in  A^  and  has  gray  value  y.  A^  essentially 
defines  a  surface  in  3-space  corresponding  to  the  gray  scale  function.  A^  can  be 
converted  into  a  binary  3-dimensional  image  called  Umbra.  The  umbra  of  A^  is  the 
space  below  the  surface,  it  can  be  expressed  as  [Sternberg]; 
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U(>l2)  =  i/3  =  I  <  m,y)  e  A^}  (31) 

The  gray  scale  image,  its  surface  and  umbra  are  illustrated  in  Figure  2. 4.6-5  for  a 
one-dimensional  image. 

For  a  two-dimensional  gray  scale  image,  its  three-dimensional  umbra  can  be 
further  decomposed  into  slices  of  two-dimensional  binary  images: 


=  U  A 

n-l.N 


Three-dimensional  morphological  operations  can  be  performed  by  operating  on 
two-dimensional  slices  and  recomposing  the  image.  For  example,  to  perform  a  dilation 
iP  ®  three  dimensional  umbra  is  first  decomposed  into  slices  in  the  y  direction. 

Operating  on  the  2-D  slices,  an  intermediate  result: 


=  U  Ay  e  R 


is  obtained.  Then,  turning  to  the  orthogonal  direction,  we  have: 


®  ^  U  ®  r\ 

where  R^y  and  denote  the  two-dimensional  slicers  of  the  three-  dimensional  structure: 
element  projected  onto  the  i-x  and  i-y  planes  respectively. 


2.4.6.3.2  Threshold  Decomposition 


The  processing  approach  described  above  is  quite  complex,  requiring  electronic 
buffer  memory  to  hold  the  intermediate  step  B^.  Threshold  decomposition  offers  a 
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simpler  approach  to  gray  scale  morphology  [Shih].  It  can  be  shown  that  gray  scale 
morphological  operation  is  the  summation  of  a  series  of  unioned  binary  operations.  If 
the  structure  element  is  binary,  the  processing  is  reduced  to  the  summation  of  a  series 
of  binary  morphological  operations  on  the  gray  scaled  image  which  has  been  decomposed 
into  a  set  of  binary  images  according  to  intensity  levels.  The  processing  for  the 
decomposed  images  can  be  performed  by  the  optical  processor  in  parallel.  Such  a 
plane-parallel  architecture  allows  the  processor  to  maintain  real-time  processing 
performance  with  gray  scale  input  images.  Alternatively,  the  decomposed  images  can 
be  processed  in  serially,  using  a  detector  or  an  SLR  to  sum  the  output  in  order  to 
minimize  hardware  complexity. 

To  make  the  results  of  threshold  decomposition  consistent,  the  shading  or  slowly 
varying  bias  of  the  gray  scale  image  should  first  be  removed.  This  can  be  achieved  with 
the  same  optical  processor  by  using  the  storage  and  gray  scale  properties  or  the  SLR. 
Let  G(x,y)  be  the  two-dimensional  gray  scale  input  image.  The  slowly  varying  bias  can 
be  estimated  by: 

S(x,y)  =  G(x,y)  *  |h(x,y)|^  (35) 

where  |  h(x,y)  |  ^  the  point  spread  function  of  an  incoherent  imaging  system  which 
performs  a  low  pass  filtering  operation.  The  modified  gray  scale  image  is  given  by 
M(s,y)  =  G(x,y)  -  S(x,y)  which  can  be  sliced  at  different  intensity  levels  into  a  set  of 
two  dimensional  binary  images. 

2.4.7  Multispectral  Optical  Preprocessor 

Many  targets  have  spectral  reflectance  characteristics  that  are  different  from  those 
of  the  clutter  background  that  can  be  used  to  aid  target  detection.  Even  with  camouflage 
paints  and  markings  designed  to  make  the  target  blends  in  with  the  background,  there  are 
measurable  differences  in  their  finer  spectral  structures.  For  example,  background  clutter 
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has  chlorophyll  and  water  absorption  lines  that  are  hard  to  emulate  accurately  with  man¬ 
made  materials.  Similarly,  synthetic  material  and  paints  may  reflect  particularly  strongly 
or  weakly  at  certain  wavelengths.  The  target-to-clutter  ratio  can  be  enhanced  by 
comparing  the  relative  spectral  reflectance  at  two  different  wavelengths.  As  an  example, 
Figure  2.4.7- 1  shows  the  spectral  reflectance  of  a  fictitious  target  and  the  background 
clutter  over  certain  spectral  region.  The  spectral  reflectance  of  the  target  is  measurably 
different  between  wavelengths  Xj  and  X2  but  for  the  clutter,  they  are  about  the  same. 
If  the  image  intensity  acquired  at  wavelength  X2  is  subtracted  from  image  taken  at 
wavelength  X^  ,  an  enhanced  image  of  the  target  is  obtained.  That  is; 

^1)  -  ^2)  '  ^2)  (36) 

=  Xi)  -  Ij(x,y;  X2). 

where  I^(x,y)  is  the  intensity  distribution  of  the  resulting  image,  Ij{x,y;  X)  is  the 
intensity  of  the  target  at  X  and  I(^x,y)  is  the  intensity  of  the  clutter.  With  the  specific 
spectral  reflectance  and  the  choice  of  the  two  wavelengths,  the  background  clutter  is 
largely  removed. 

2  4.7.1  Optical  Processing  Architectures  for  Multispectral  Preprocessor 

The  ability  of  an  SLR  to  perform  subtraction  can  be  used  as  multispectral 
preprocessor.  A  possible  optical  processing  architecture  is  illustrated  in  Figure  2. 4. 7-2. 
As  explained  in  Section  1,3,  a  passive  SLR  is  written  and  read  out  at  the  two  different 
wavelengths.  As  a  multispectral  preprocessor,  an  SLR  with  write-in  wavelength  of  Xj 
and  readout  wavelength  of  X2  is  used  as  the  input  device  for  the  imaging  sensor  with  a 
two-band  spectral  filter  centering  at  Xj  and  X2  .  For  day-night  operation,  the  object 
scene  can  also  be  actively  illuminated  with  two  lasers  emitting  at  Xj  and  X2  .  The 
relative  brightness  of  the  clutter  image  at  Xj  and  X2  is  adjusted  such  that  the  rate  of  build 
up  of  trapped  electrons  by  wavelength  Xj  is  about  the  same  as  the  rate  depletion  by 
wavelength  X2  .  Under  such  a  condition,  the  number  of  trapped  electrons  due  to  the 
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Figure  2  4  7-1  Multispectral  Target-to-Clutter  Ratio  Enhancement 
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P'lgure  2  4  7-2  Multispectral  Optical  Processor 
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target  image  will  increase  with  time  since  the  target  is  brighter  at  Xj  .  A  clutter 
suppressed  image  of  the  target  can  be  obtained  by  simply  reading  the  image  in  the  SLR 
out  with  a  uniform  readout  beam  at  wavelength  X2. 

If  the  target  image  is  brighter  at  the  readout  wavelength,  instead,  the  SLR  can 
first  be  flooded  with  light  at  wavelength  Xj  to  saturate  the  SLR.  The  number  of  trapped 
electrons  due  to  the  image  will  reduce  with  time  and  the  result  is  an  enhanced  and 
contrast  reversed  image  of  the  target. 

2.5  Down  Selection 

A  pre-selection  was  performed  that  resulted  in  seven  candidates  for  evaluation. 
Of  the  seven,  five  were  down  selected  for  further  analyses.  They  are  the  artificial  neural 
network,  the  quadratic  processor,  the  morphological  processor,  the  interferometric 
processor  and  the  OTF  synthesis  optical  preprocessor.  Reasons  for  their  selection  are 
given  below. 

Unlike,  for  example,  a  Fourier  transform-based  processor,  an  artificial  neural 
network  (ANN)  operates  from  a  low  level  which  makes  it  the  most  versatile  of  the 
architectures  studied.  It  can  be  used  to  implement  nearly  all  the  functional  elements  in 
ATC.  The  basic  operation  required  is  sum  of  products  and  thresholding  which  can  be 
performed  very  efficiently  with  an  optical  processor.  In  addition,  the  massive  fan-outs 
in  an  ANN  architecture  can  be  accomplished  more  easily  with  optical  interconnects  than 
with  electrical  wires. 

A  quadratic  processor  performs  pixel-by-pixel  statistical  target  detection  on  the 
target  scene.  It  utilizes  local  spatial  variations  (reflectance  or  emittance)  as  a 
discriminant  between  targets  and  clutter.  The  processing  architecture  can  utilize  the 
incoherent  sensor  image  directly  as  the  input.  It  allows  the  quadratic  processor  to  bypass 
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the  limitations  of  optical  to  electrical  and  electrical  to  optical  converters,  making  it 
particularly  useful  as  a  preprocessor.  In  addition,  the  nonlinear  operations  performed  by 
a  quadratic  processor  provide  processing  capabilities  not  available  with  conventional 
linear  optical  systems. 

The  morphological  processor  is  also  a  neighborhood  processor  capable  of 
nonlinear  processing  functions.  Instead  of  matching  to  the  overall  shape  of  a  target,  a 
morphological  processor  performs  ATC  by  extracting  features  that  define  a  target.  Such 
a  process  tends  to  be  more  robust  than  matched  filtering  or  template  matching  whose 
performance  can  be  adversely  affected  by  changes  in  aspect,  lighting  and  operating 
condition.  Morphological  processing  is  typically  implemented  electronically  using  a 
recirculating  pipeline  architecture  to  minimize  the  size  of  the  processor.  With  the 
inherent  parallelism  of  an  optical  processor,  the  neighborhood  operation  can  be 
perfonned  simultaneously  for  all  pixels  in  the  image. 


The  last  two  are  diffraction  based  incoherent  optical  processors  whose 
architectures  are  well  known.  Diffraction  based  systems  provide  the  largest  space- 
bandwidth  product,  making  them  attractive  for  wide  area  search  applications.  The 
difficulty  has  been  with  the  bias  which  could  easily  overwhelm  the  signal  at  the  output. 
With  both  architectures,  the  role  of  the  SLR  is  bias  reduction.  Since  the  principles  of 
these  two  types  of  incoherent  optical  proces'wrs  are  well  established,  the  discussion  in 
Section  3  will  concentrate  on  the  use  of  a  passive  SLR  for  bias  reduction. 

The  scanning  correlator  was  not  chosen  because  of  the  its  limited  capability  (the 
reference  function  must  be  real  and  positive)  and  relatively  slow  speed  due  the  serial 
nature  of  the  scanning  operation.  The  multispectral  optical  preprocessor  did  not  survive 
the  down  selection  because  the  laser  power  required  for  flood  illumination  may  be  too 
high  to  be  practical  in  view  of  the  sensitivity  of  available  passive  SLRs.  For  example, 
to  illuminate  and  image  a  120  m  x  120  m  area  which  has  an  average  reflectance  of  0.2 
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from  a  distance  of  1  km  using  a  laser  source  with  P  watts  of  power  and  an  f/2  lens  with 
a  4  cm  aperture,  the  image  intensity  on  the  SLR  will  be  about  3.2  x  10'^^  watls/cm^. 
The  Quantex  SLR  requires  about  10  mJ/cm^  of  exposure  energy  to  reach  saturation. 
With  an  exposure  time  of  0. 1  second,  P  =  3. 1  x  10®  watts.  The  laser  has  to  emit  over 
300  Mwatts  of  power.  Moreover,  discrimination  between  different  sets  of  target  ana 
clutter  may  involve  different  combinations  of  wavelengths.  To  reprogram  the 
preprocessor  will  require  a  change  in  the  SLR  material  to  one  with  different  input  and 
readout  wavelengths. 
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3.0  TASK  2:  IN-DEPTH  ANALYSES 


The  candidate  optical  processing  architectures  were  down  selected  in  Task  2  to 
five:  1)  Artificial  Neural  Network,  2)  Quadratic  Processor,  3)  Morphological  Processor, 
4)  OTF  Synthesis  Preprocessor  and  5)  Interferometric  processor.  In  this  section,  the 
design  of  the  selected  architectures  are  analyzed  in  greater  depth  to  evaluate  their 
potential  performance  and  viabilities. 

3.1  Evaluation  Criteria 

The  SLR-based  incoherent  optical  processors  have  to  compete  with  well 
entrenched  electronic  processors  and  with  coherent  optical  processors  as  well  as  with 
each  other.  The  primary  performance  figure  for  processor  comparison  is  computation 
speed  or  system  throughput  in  terms  of  the  number  of  operations  per  second.  However, 
for  Air  Force  ATR  missions,  the  sensors  and  processors  have  to  be  carried  on  aircraft, 
unmanned  aerial  vehicles  and  missiles.  The  processor  size  and  weight  become  important 
issues.  Electronic  processors  can  achieve  processing  speed  as  fast  or  faster  than  any 
optical  processor  by  implementing  massive  amount  of  parallelism.  Such  a  feat  is 
accomplished  at  the  expense  of  processor  size.  A  better  performance  figure  for 
comparison  is  throughput  per  unit  volume  (e.g. ,  operations  per  second  per  cn?).  Power 
consumption  is  also  important  issue,  particularly  for  satellite  borne  processors.  Rough 
estimates  of  the  power  requirements  of  the  optical  processors  are  also  provided.  Beside 
the  ultimate  potential  performance,  the  near  term  availability  of  optical  components  for 
the  optical  processing  architectures  were  also  investigated  to  determine  the  viability  of 
the  optical  processing  architectures. 
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3.2  Artificial  Neural  Network 

In  Section  2.4.3,  five  architectures  for  artificial  neural  networks  were  discussed. 
The  cylindrical  optics  architecture  with  a  fixed  weight  mask  was  chosen  for  ir. -depth 
analysis.  An  architecture  allowing  real-time  optical  modification  of  the  weights  was  also 
identified  as  a  candidate  for  further  analysis.  The  in-depth  analysis  began  with  a  study 
of  computing  accuracy  requirements  and  led  to  a  decision  that  optical  processing  is 
neither  suited  for  nor  needed  for  weight  computation.  The  cylindrical  optics  architecture 
was  then  analyzed  in-depth.  It  was  found  that  the  cylindrical  optics  architecture  requires 
some  significant  modifications.  A  new  opto-electronic  architecture  based  on  integrated 
optics  was  then  developed,  analyzed,  and  shown  to  be  superior  in  terms  of  size  required 
while  not  sacrificing  performance  in  any  other  area. 

3.2.1  Accuracy  and  Real-Time  Computation  Requirements 

Optical  processing  generally  has  less  accuracy  than  electronic  digital  processing 
but  greater  speed.  In  other  ERIM  work,  a  study  was  done  to  determine  the  accuracy 
required  for  artificial  neural  network  computations.  That  study  is  briefly  summarized 
here  [Cederquist  et  al].  The  problem  chosen  was  that  of  determining  terrain  type  (forest, 
grass,  soil)  from  airborne  sensor  imagery  of  the  ground  in  five  wavelength  bands  in  the 
visible,  near  infrared,  and  short  wave  infrared.  A  Kohonen  self-organizing  network  was 
successfully  developed  for  this  purpose.  The  network  has  five  inputs,  three  nodes,  and 
three  outputs  corresponding  to  the  terrain  classification.  The  network  was  trained  with 
floating  point  computation.  The  network  was  then  used  to  classify  the  input  data  with 
varying  accuracy  in  the  input  data  and  weights.  The  results  are  shown  as  confusion 
matrices  in  Fig.  3.2-1.  The  result  is  that  floating  point  performance  is  maintained  down 
to  5-bit  accuracy  in  the  data  and  weights.  Although  this  is  only  a  single  test,  it  was 
assumed  that  6-bit  accuracy  is  sufficient  for  artificial  neural  network  computations  during 
use,  but  not  during  training. 
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Figure  3.2-’.  Confusion  Matrix  for  Test  Data  with  Finite  Weight  and  Input  Precision 
(a)  Floating-point  Weight  and  Input  Precision;  (b)  6-bit  Weight  and  Input 
Quantization;  (c)  5-bit  Weight  and  Input  Quantization 
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optical  computation  of  weight  values  was  then  considered.  From  ERIM 
experience  with  perceptron  and  Kohonen  artificial  neural  networks  for  automatic  target 
classification  applications  such  as  edge  enhancement  (ship  wakes  in  SAR  imagery), 
segmentation  (terrain  classification  from  optical  multispectral  imagery),  and  object 
detection  (mine  location  in  laser  imagery),  it  is  known  that,  during  training,  the  changes 
in  the  weights  at  each  cycle  require  much  more  than  6-bit  accuracy.  Optical  processing 
in  general  and  the  architecture  discussed  in  Section  2.4.5  in  particular  have  never  been 
shown  to  be  capable  of  this  greater  accuracy.  As  an  example,  the  few  optical  processors 
which  were  carefully  tested  for  accuracy  generally  achieved  about  5  %  [Cederquist  and 
Lee]. 


The  need  for  real-time  computation  of  weights  was  also  critically  analyzed.  For 
the  automatic  target  classification  application,  which  is  one  application  of  interest  to  the 
Air  Force,  it  is  very  difficult  to  argue  that  weights  would  be  computed  in  real-time,  i.e., 
during  an  Air  Force  mission.  Air  Force  doctrine  requires  pre-mission  planning  and  high 
confidence  in  the  success  of  the  mission.  The  automatic  target  classification  algorithms 
used  would  need  to  be  known  and  tested  for  performance  before  the  mission.  If  artificial 
neural  networks  were  to  be  trained  during  the  mission,  there  is  no  current  knowledge  that 
guarantees  that  training  would  be  successful  and  no  current  method  of  measuring 
performance  other  than  comparison  with  targets  and  backgrounds  identified  by  the  air 
crew  during  the  mission.  In  short,  artificial  neural  network  training  algorithms  are  not 
yet  sufficiently  developed  to  be  used  during  a  mission,  so  there  is  not  a  compelling 
reason  for  pursuing  optical  computation  of  artificial  neural  network  training  algorithms. 

3.2.2  Cylindrical  Optics  Architecture  Analysis 

Two  versions  of  the  cylindrical  optics  architecture  were  designed  using  the  GENII 
lens  design  software  and  analyzed.  The  first  is  shown  in  Fig.  3.2-2.  A  laser  diode  array 
was  chosen  for  the  linear  input  device.  In  a  cascaded  version,  this  would  be  the  output 
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of  the  spatial  light  rebroadcaster.  An  array  of  cylindrical  (piano-elliptical  cross-section) 
lenses  is  used  to  collimate  the  asymmetrical  beams  from  the  lasers  in  one-dimension. 
The  first  cylinder  lens  is  elliptical-piano  in  cross-section  and  provides  collimated  beams 
at  the  weight  mask.  The  second  cylinder  lens  is  piano-hyperbolic  in  cross-section  and 
focuses  the  beams  onto  a  linear  photo-diode  array  (input  side  of  spatial  light 
rebroadcaster).  Additional  raytraces  are  shown  in  Fig.  3.2-3.  The  mask  layout  is  shown 
in  Fig.  3.2-4  for  cases  where  the  number  of  nodes  per  network  is  small  and  multiple 
networks  can,  therefore,  be  computed  simultaneously.  Positive  and  negative  weight 
channels  are  needed  because  of  intensity  detection.  The  point  design  of  this  architecture 
was  completed  by  specifying  the  following: 

Input:  100  laser  diodes  at  1  mm  spacing  and  1  MHz  operation  rate,  Mask:  1(X)  by  200 

1mm  by  250  micron  pixels 

Output:  200  photodiodes  at  250  micron  spacing  and  1  MHz  operation  rate. 

This  leads  to  the  following  characteristics: 

Volume:  L  by  W  by  H  =  20  cm  by  10  cm  by  5  cm  -  1000  cm^ 

Input  data  rate:  1(X)  x  1  MHz  =  10®  data  values/sec 
Computation  rate:  100  x  200  x  1  MHz  =  2  x  10*°  operations/ sec 
Computations/cm^:  2  x  10^ 

Power  (for  1  %  accuracy):  100  x  40  mW  =  4  W 

The  main  disadvantage  of  this  first  architecture  is  the  large  volume  resulting  in 
a  relatively  low  number  of  computations  per  unit  volume.  This  large  required  volume 
is  not  generally  recognized  in  the  optical  processing  literature  because  other  researchers 
have  not  attempted  to  design  an  actual  optical  system  use  optical  raytracing  design 
software.  The  literature  generally  shows  only  very  simplified,  schematic  drawings  of 
cylindrical  optics  architectures  usually  for  matrix-vector  multipliers. 
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Figure  3.2-4.  Mask  Layout  for  an  Optical  Artificial  Neural  Network 
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In  an  effort  to  reduce  the  processor  volume,  a  second  cylindrical  optics 
architecture  was  developed  and  analyzed.  The  architecture  is  shown  in  Figure  3.2*5. 
Again,  a  laser  diode  array  was  chosen  for  the  linear  input  device.  A  pair  of  cylinder 
lenses  with  an  aperture  stop  is  used  to  transfer  the  input  to  the  weight  mask  by  imaging 
it  in  one-dimension  and  collimating  it  in  the  other.  Again,  the  GENII  optical  design 
software  was  used  in  our  analysis.  The  mask  layout  is  identical  to  that  for  the  first 
architecture,  shown  in  Figure  3.2-4.  The  main  difference  of  this  architecture  from  the 
first  is  that  a  linear  array  of  long,  narrow  photodiodes  is  used  to  electrically  collect  the 
light  passing  through  the  mask  rather  than  using  optics  to  focus  the  light  onto  a  linear 
array  of  point  detectors.  Using  this  mode  of  operation  means  that  the  light  at  the  mask 
does  not  need  to  be  collimated  and  thereby  reduces  the  constraints  on  the  cylindrical  lens 
design.  This  in  turn  allows  a  more  compact  design.  A  1  MHz  readout  rate  is  assumed 
for  both  architectures  so  the  computation  rate  is  not  decreased.  The  point  design  of  this 
architecture  was  completed  by  specifying  the  following: 

Input:  100  laser  diodes  at  100  micron  spacing  and  1  MHz  operation  rate 

Mask:  100  by  200  100  micron  by  50  micron  pixels 

Output:  200  linear  photodiodes  at  50  micron  spacing  and  1  MHz  operation  rate 


This  leads  to  the  following  characteristics: 

Volume;  L  by  W  by  H  =  6  cm  by  1  cm  by  1  cm  =  6  cm^- 
Input  data  rate;  100  x  1  MHz  =  10®  data  values/sec 
Computation  rate:  100  x  200  x  1  MHz  =  2  x  10*®  operations/sec 
Computations/cm^;  3  x  10^ 

Power  (for  1  %  accuracy):  100  x  40  mW  =  4  W 

This  second  architecture  is  superior  to  the  first  in  computation/cm^  as  desired. 


It  was  found  in  both  these  designs  that  the  mask  two-dimensional  space-bandwidth 
product  was  proportional  to  the  square  root  (W  x  H).  The  two  designs  have,  of  course, 
different  proportionality  constants.  This  result  is  not  surprising.  It  is  well  known  in  the 
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Figure  3.2-5.  Alternate  Cylindrical  Optics  Architecture  for  an  Artificial  Neural  Network 
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optical  processing  literature  that  optical  systems  which  can  implement  arbitrary  optical 
interconnections  of  the  input  to  the  output  (e.g.,  cross  bar  switches)  have  their 
space-bandwidth  product  limited  by  diffraction  effects  to  the  square  root  (W  x  H).  To 
overcome  this  fundamental  limit,  a  new  optical  architecture  was  developed  and  analyzed. 

3.2.3  Integrated  Optics  Architecture  Analysis 

The  integrated  optics  architecture  is  shown  in  Figure  3.2-6.  Once  again,  a  laser 
diode  array  was  chosen  for  the  linear  input  device.  The  lasers  are  coupled  into  a  linear 
array  of  optical  multimode  waveguides,  one  laser  for  each  waveguide.  Each  waveguide 
has,  along  its  length,  a  number  of  devices  which  couple  light  out  of  the  waveguide. 
Either  by  controlling  the  amount  of  light  coupled  out  or  by  placing  a  mask  next  to  the 
waveguide  array,  the  input  light  is  multiplied  by  the  desired  artificial  neural  network 
weights.  This  light  is  then  electrically  collected  by  a  linear  array  of  long,  narrow 
photodiodes  similar  to  that  used  in  the  second  cylindrical  optics  architecture.  Processing 
of  positive  and  negative  channels  is  done  electronically.  The  results  can  be  rebroadcast 
to  the  next  layer  by  electrical  connection  to  another  laser  diode  array.  Both  of  these 
functions  could  be  done  with  integrated  electronics  to  maintain  a  compact  artificial  neural 
network  processor. 

The  technology  required  for  the  implementation  of  this  processor  is  currently 
available  (laser  diode  arrays,  optical  waveguides,  linear  photodiode  arrays)  or  an 
extension  of  current  electronic  technology  (integration  of  photodiodes  with  operational 
amplifiers  and  laser  diodes)  except  for  the  method  of  coupling  light  out  of  the 
waveguides.  Possible  methods  for  achieving  this  coupling  are  (1)  transparent  windows 
in  otherwise  totally  reflecting  waveguide  walls  which  would  let  light  escape,  (2)  diffuse 
regions  in  the  walls  which  would  scatter  light  out,  and  (3)  grating  couplers  which  would 
diffract  light  out  of  the  waveguide.  This  methods  are  discussed  further  in  Section  4.3. 
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Figure  3.2-6.  Integrated  Optics  Architecture  for  Artificial  Neural  Network 
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The  point  design  of  this  architecture  was  completed  by  specifying  the  following: 

Volume:  L  by  W  by  H  =  0.5  cm  by  1  cm  by  1  cm  =  0.5  cm^  (where  L  is 
measured  perpendicular  to  the  plane  of  the  waveguides) 

Input  data  rate:  100  by  1  MHz  =  10®  data  values/sec 
Computation  rate;  100  x  200  x  1  MHz  —  2  x  10*®  operations/sec 
Computations/cm^;  4  x  10*® 

Power  (for  1  %  accuracy):  100  x  40  mW  =  4  W 

The  length  L  (or  more  properly  device  thickness)  is  only  an  estimate  based  on 
integrated  electronic  flip  chip  and  bump  bonding  practices.  The  integrated  optics 
architecture  has  about  a  factor  of  10  greater  computations/cm^.  It  should  also  be  noted 
that  the  space-bandwidth  product  is  proportional  to  W  x  H.  This  means  that,  if  it  were 
desired  to  scale  the  space-bandwidth  product  of  the  point  designs  presented  in  this  section 
up  by  a  factor  of  2  in  each  dimension,  the  integrated  optics  architecture  would  grow  by 
a  factor  of  4  in  volume  while  the  cylindrical  architectures  would  grow  by  16.  Other 
advantages  of  the  integrated  optics  architecture  are  (1)  that  it  builds  on  electronic 
microfabrication  technology  and  will  benefit  from  advances  in  that  arena  and  (2)  that  the 
device  is  a  single  component  which  should  be  more  rugged  and  easier  to  keep  in 
alignment  than  the  cylindrical  optics  architecture  with  its  multiple,  separated  optical 
components. 

Some  initial  concepts  were  developed  for  further  integration  of  the  integrated 
optics  architecture.  First,  multiple  layers  of  an  artificial  neural  net  could  be  integrated 
onto  a  single  substrate  as  shown  in  Figure  3.2-7.  A  two-dimensional  spatial  light 
rebroadcaster,  such  as  the  three  terminal  device  being  developed  by  AT&T  could  then 
be  used  as  the  optical  to  electrical  to  optical  convertor  and  input-output  device. 
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Data  Output 


Figure  3.2-7.  Multiple  Layer  Integrated  Optics  Implementation  for  Artificial  Neural 
Network 


3.3  Quadratic  Processor 


The  primary  function  of  the  quadratic  optical  processor  (shown  in  Figure  3.3*1) 
is  to  implement  a  pixel-by-pixel  statistical  target  detection  algorithm  applied  to  2D  optical 
scenes.  This  optical  processor  would  be  a  pre-delection  processor  whereby  "detection" 
signifies  the  detection  of  an  extended  multipixel  target.  The  main  attributes  of  this 
architecture  is  the  total  parallelism  in  its  implementations  (all  pixels  are  processed 
simultaneously)  and  the  lack  of  optical/electronic/optical  interfaces.  No  arbitrary 
detection  and  spatial  light  modulation  is  incorporated  in  the  architecture. 

An  understanding  of  tlie  salient  features  of  the  optical  architecture  can  be 
facilitated  through  a  detailed  analysis  of  the  quadratic  algorithm.  The  quadratic  operation 
used  in  the  pixel-by-pixel  detector  is: 

L^fAf>T  (37) 

where /is  the  feature  vector  under  consideration  and  A  is  proportional  to  the  inverse  of 
the  covariance  matrices  and  T  denotes  transpose.  The  feature  vector/  will  be  assumed 
to  be  the  image  data  over  a  local  3x3  neighborhood.  The  covariance  matrix  A  can  be 
calculated  off-line  through  training  data. 

The  optical  realization  of  matrix  vector  operations  is  through 
coefficient-by-coefficient  multiplication  and  summation.  The  first  part  of  the  quadratic 
operation  f^A  can  be  written  as: 

=/^/l  =  [gi,...,  .  SnI 

Note  that  each  coefficient  gjj  is  produced  by  the  multiplication  of  each  member  of  the 
row  vector/^  with  the  members  of  the.  specific  column  of  A  and  then  summed  cs; 
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Figure  3.3-1.  Optica]  Quadratic  Processor 
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N 

Sn~^^yi  ^i,n  • 

1  =  1 

Note  that  the  are  replicated  for  each  column  n  in  .4  producing  a,  „  .  Laftly,  the 
vector  inner  product  g^f  is  computed  as: 

i=l 


Recall  that  the  input  data  are  produced  by  an  optical  imaging  sensor.  The  feature 
vector  is  defined  as  the  local  3x3  neighborhood  around  a  specific  pixel.  This  physical 
situation  requires  an  altered  lexicographic  numbering  of  the  input  vector  f  and  filtering 
array  A.  Instead  of  the  input  vector  having  the  form  •••.fn,  •••./p]  ,  the  input 

vector  will  be  ordered  as: 

A  fl  A 

f  =  A  fs  fe  • 

/?  /g  f9 

which  is  what  is  physically  present  in  the  optical  system.  By  allowing  this  numbering 
convention,  no  additional  special  optics  need  be  applied  to  display  the  vectors  in  the 
conventional  way. 

The  first  lenslet  array  selects  the  local  3x3  neighborhood  f.  As  shown  in  Figure 
3.3-2,  each  pixel  in  the  scene  is  contained  in  9  neighborhoods.  Its  own  and  that  of  the 
8  neighbors.  Figure  3.3-2  shows  how  the  lenslet  array  isolates  each  neighborhood.  Each 
lens  in  the  array  has  an  overlapping  field-of-view  where  two  columns  (or  two  rows  for 
vertically  oriented  lenslets)  of  the  3  x  3  area  are  shared  between  two  horizontally  spaced 
lenslets.  In  order  for  this  system  to  operate,  three  conditions  must  be  met:  1)  the  number 
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Figure  3.3-2.  Selection  of  Neighborhood  with  Lenslet  Array 


Figu.e  3.3-3.  Replication  of  Local  Neighborhood  with  Lenslet  Array. 
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of  lenslets  in  the  array  must  be  equivalent  to  the  number  of  pixels  in  the  input  image. 
This  is  due  to  the  fact  that  the  number  of  neighborhoods  is  equivalent  to  the  number  of 
pixels.  2)  the  lenslet  size  must  be  less  than  or  equal  to  the  pixel  size  of  the  input  image. 
When  equal,  the  light  throughput  of  each  lenslet  is  maximum.  The  input  pixel  size  will 
be  assumed  to  be  approximately  200  fim  (consistent  with  the  SLR  pixel  size 
specification).  3)  When  imaged,  the  local  neighborhood  will  be  minified  by  the  lenslet 
array.  This  requirement  attempts  to  control  crosstalk  for  the  next  lenslet  array  which 
replicates  f. 

The  second  lenslet  array  replicates  the  local  neighborhood  as  shown  in  Figure 
3.3-3.  This  step  is  required  to  affect  the  vector  matrix  product  f^A  as  outlined  above. 
The  input  vector  (the  local  neighborhood)  is  replicated  and  multiplied  by  the  appropriate 
column  in  the  matrix  A.  The  basis  for  this  step  is  the  use  of  multiple  lenslets  with 
completely  overlapping  fields-of-view  to  reimage  the  local  neighborhood  numerous  times. 
In  our  case  9  lenslets  are  required  per  neighborhood  replicating  the  same  neighborhood 
9  times. 

The  multiplication  of  the  replicated  vector /  by  the  matrix  A  is  accomplished  via 
point-by-point  multiplication  through  a  mask  (either  film  or  SLR).  The  matrix  A  must 
also  be  lexigraphically  reordered  as  described  previously  and  shown  in  Figure  3.3-4. 
Once  reordered,  a  simple  incoherent  multiplication  is  affected  by  the  mask.  The  final 
summation  which  then  produces  the  output  g=/^^  is  provided  by  the  next  lenslet  array. 
In  this  system,  each  lenslet  has  distinct  fields-of-view  which  are  the  neighborhoods  in 
their  entirety  (see  Figure  3.3-5).  A  simple  integration  via  focusing  is  then  accomplished. 
The  number  of  lenslets  in  this  array  correspond  to  the  number  of  neighborhoods. 

The  output  g=/  A  is  incident  on  the  next  SLR  which  also  has  the  radiation  of 
the  replicated  neighborhood  impinging  upon  it  from  the  other  side.  This  neighborhood 
data  are  simply  the  output  of  the  sensor  split  from  the  main  path  of  the  quadratic  system 
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via  a  beamsplitter  as  shown  in  Figure  3.3-1.  These  data  have  their  neighborhoods 
selected  via  a  lenslet  array  as  discussed  previously.  The  same  conditions  apply  to  the 
lenslet  array  as  previous  (number  of  lenslet  are  equal  to  the  number  of  input  points).  A 
second  lens  (non-lenslet)  reimages  the  neighborhoods  onto  the  SLR  via  a  second 
beamsplitter.  A  point-by-point  multiplication  of  g  is  then  produced  by  the  SLR. 

The  final  summation  of  the  terms  in  gf^  to  produce  the  output  scalar  is  provided 
by  the  last  lenslet  array  of  the  system.  Each  lenslet  in  the  array  has  a  distinct 
field-of-view  and  provides  the  summation  operation  through  focusing  the  energy  onto  a 
spot.  The  number  of  lenslets  in  this  array  is  equal  to  the  input  number  of  points.  The 
output  spot  must  be  small  enough  such  that  the  finite  size  of  the  detection  elements  of  the 
SLR  completely  cover  the  focused  spot.  Lastly,  the  final  SLR  is  used  as  a  means  of 
setting  a  threshold  on  the  incident  radiation  producing  the  final  binarized  output. 

We  will  now  embark  on  a  detailed  analysis  of  the  system  size  and  the 
specifications  on  the  optical  components.  This  analysis,  in  conjunction  with  Section 
2.3.2  will  allow  us  to  completely  determine  the  system  throughput  density  (ops  per 
volume  per  unit  power)  which  is  the  performance  metric  useful  for  a  comparison  to 
digital  systems.  In  our  analysis  we  will  rely  on  system  geometry  and  simple  imaging 
equations  (lensmakers  equation,  F-number,  magnification,  and  Rayleigh  resolution 
criterion)  to  drive  our  analysis.  In  addition,  we  will  also  design  the  system  to  fall  within 
well  accepted  specifications  on  the  lenslet  arrays  described  in  [Borelli]  and  summarized 
here: 

Lenslet  diameter:  lOntn  <  D  <  lOOO/xm 

Lenslet  spacing:  15/am  <  Delta 

Focal  lengths:  2()0/am  <  f  <  50mm 

F-number:  F^  >  1.43 


98 


To  begin  our  analysis,  we  will  define  the  lensmakers  equation  as: 

(41) 

where  /  is  ti:e  lens  focal  length,  is  the  object  distance  and  is  t.ie  image  distance. 
The  magnification  factor  is  given  as: 

M=-Si  /s^=yi  /y„  (42) 

where  y,-  is  the  image  size  and  y^  is  the  object  size.  In  order  to  facilitate  the  analysis  in 
terms  of  lens  diameters,  spacings,  etc.,  we  will  define  sizes  in  terms  of  the  lens  diameter 
as  D,  y,=^2  ^  calculate  the  object  and  image 

disttJices  in  terms  of  the  constants  and  ^2 

IM=f(k^  Jk2*1).  (42) 

The  last  required  parameter  for  the  analysis  is  the  minimum  resolvable  spot  size.  This 
parameter  defines  the  lenslet  diameter.  We  will  use  the  Rayleigh  criterion  which  states 
5  =  i.22  X  Sj  /D  where  X  is  the  wavelength  of  the  radiation. 

The  parameters  of  each  lenslet  array  are  influenced  by  subsequent  arrays  in  the 
optical  system.  These  arrays  help  define  required  magnifications  (or  m’nifications).  The 
requirements  are  also  usually  defined  in  terms  of  their  lenslet  sizes  Dq  and  spacings 
We  will  now  derive  the  numerical  parameters  of  the  neighborhood  selection  lenslet  array 
in  detail.  The  subsequent  arrays  are  analyzed  in  an  equivalent  manner.  Therefore,  only 
their  results  will  be  given. 

Figure  3.3-6  shows  the  lenslet  geometry  that  will  be  used  in  this  analysis.  It  will 
be  assumed  that  the  input  pixel  spacing  is  equal  to  the  lenslet  spacing.  Since  the  field 
of  view  of  the  lenslet  must  encompass  3  input  pixels,  the  object  size  to  be  imaged  is: 
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Figure  3.3-6.  Lenslet  Geometry  in  Optical  Quadratic  Processor 


y^-  2Dj+2Aj-  AtjDj 


(44) 


where  Dj  and  Aj  are  the  lensiet  diameter  and  spacing,  respectively.  Since  the  object 
consists  of  only  3  points,  the  minimum  resolvable  spot  separation  at  the  output  is: 

5  =  ;y.  /3  =  A:2Dj  /3  =  1.22X5,  ID  (45) 


which  leads  to: 


Z)i  >  ^3.66X/(l/yt2+l/iti)  . 

From  the  geometry  of  Figure  3.3-6: 

Z)j+Aj=  3D2'*'^^2‘ 

We  will  also  assume  symmetry  such  that  3D2  =  D  and  2A2  =  A,  In  order  to  minimize 
crosstalk  between  the  lenslets  in  the  replication  stage  y^  <  D2  =  Dj/3  which  leads  to 
k2  <  1/3.  We  will  use  as  our  baseline,  a  lensiet  array  with  Dj  =  300  urn  and 
Ai  =  45  /xm  which  is  within  the  specifications  of  [Borelli].  From  this,  we  can  easily 
derive  that  the  focal  length  of  the  first  lensiet  array  must  be  f  =  8.9  mm  with  an  F#=30. 
The  lensiet  array  must  be  a  distance  of  s^  =  7cm  from  the  object  and  its  formed  image 
is  a  distance  of  Sj  =  1.02  cm  from  the  array.  These  results  are  shown  in  Figures  3.3-6 
and  3.3-7.  A  similar  set  of  derivations  can  be  performed  for  the  neighborhood 
replication,  summing  over  the  filtered  output,  and  summing  over  the  second  inner 
product  lensiet  arrays.  The  results  of  these  calculations  are  shown  in  Figure  3.3-6  with 
the  overall  system  size  (and  object/image  distances)  shown  in  Figure  3.3-7. 

The  system  power  is  defined  by  the  final  active  SLR,  all  the  other  components 
are  passive.  The  SLR  required  power  is  assumed  to  be  3mW  per  pixel.  The  total  power 
then  becomes  3mW  x  (500)^  =  750  Watts  for  the  entire  system.  The  power  per  unit 
volume  is  then  18mW/cm''3  (based  on  the  calculated  41,000  cm^  processor  volume). 


(46) 


(47) 
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Figure  3.3-7.  Geometry  of  Optical  Quadratic  Processor  used  for  Evaluation 
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The  last  calculation  for  system  performance  is  the  system  signal-to-noise  ratio. 
This  determines  whether  or  not  there  are  sufficient  photons  in  the  system  to  produce  a 
usable  signal  above  the  noise  floor.  In  order  to  proceed,  we  must  assume  an  operational 
scenario.  This  scenario  was  discussed  in  Section  2.1.  We  will  just  summarize  the 
parameters  here  as: 

Sensor  range:  15  km 

Sensor  Lens  Diam.:  17  cm 
F5  system:  f=85  cm 

Scene:  Sunlight  veg. 

Fgun  =  4  X  10'^  watts/cm^/pm  (X  =  0.6/xm) 

These  specifications  produce  an  average  intensity  at  the  image  plane  of  the  sensor  as 
I  =  0.282  Watts/m^  (based  on  Tj^j^  =  0.9,  AX  =  0.5  /xm).  Figure  3.3-8  depicts  the 
assumed  transmission  coefficients,  responsivities  and  optical  power  division  and  increase 
(through  summation)  in  the  system.  In  addition,  the  optical  power  densities  are  shown 
at  different  points  along  the  system.  Lastly,  the  assume  specifications  of  the  SLR 
(quantum  efficiency,  time  constant,  and  spectral  bandwidth)  are  given.  Based  on  these 
specifications  the  signal  to  noise  (photon)  ratio  is  51.4  (17dB).  This  signal-to-noise  ratio, 
when  completely  used  by  target  energy  would  provide  a  detection  probability  of  Pj)=  .77 
with  a  false  alarm  probability  of  Pfa  =  lO"^.  This  is  predicated  on  the  use  of  Gaussian 
models  for  both  the  clutter  and  target  and  only  the  use  of  energy  as  the  discriminant 
[VanTrees]. 

As  noted  in  Section  2.3.2,  the  system  throughput  is  related  to  the  SLR  response 
time.  The  best  case  response  is  1  /xsec.  In  this  instance  the  system  throughput  rate  is 
42,500  Gops/sec  (see  Section  2.3.2  for  analysis  details).  For  a  processor  volume  of 
41,000  cm^,  and  prime  power  of  750  watts,  the  throughput  rate  per  unit  volume  is  1 
Gop/sec  cm^  and  the  throughput  rate  per  unit  power  is  57  Gop/sec  W.  These 
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Figure  3.3-8.  Optical  Quadratic  Processor  and  Assumed  Optical  Efficiencies  Used  in 
Signal-to-Noise  Analysis 
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specifications  are  extremely  high  and  provide  a  favorable  comparison  to  state  of  the  art 
electronic  systems  [Gary], 


3.4  Morphological  Processor 

An  important  feature  of  morphological  image  processing  is  its  inherent  parallel 
nature  where  the  same  transformations  operate  on  all  pixel  elements.  Morphological 
processing  can  be  performed  at  very  high  speed  with  an  appropriate  parallel  processing 
architecture.  There  several  basic  architectures  that  can  be  used  to  implement  a 
morphological  image  processor. 

1)  Parallel  Full  Array: 

A  full  array  of  two-dimensional  processors  with  each  processor  connected  to  other 
processors  in  its  neighborhood.  All  processors  execute  the  same  instruction 
simultaneously  which  is  broadcast  from  a  central  controller.  This  type  of  Single 
Instruction  Multiple  Data  (SIMD)  processor  architecture  is  extremely  difficult  to  be 
implemented  with  conventional  microelectronics,  requiring  a  hugh  number  of  parallel 
processors  and  interconnects.  A  full  parallel  array  has  not  been  fabricated  with 
conventional  electronics. 

2)  Parallel  Subarrav: 

A  parallel  subarray  is  simply  a  portion  of  a  full  array  which  reduces  the  hardware 
requirements.  However,  to  process  the  full  image,  the  image  has  to  be  partitioned  and 
loaded  into  the  subarray  processor  sequentially  which  adds  to  system  complexity. 

Several  parallel  subarrays  processors  have  been  built,  most  notably  the  Cellular 
Logic  Image  Processor  (CLIP)  series  and  the  Massively  Parallel  Processor  (MPP). 
These  machines  are  still  quite  massive,  requiring  many  processors  and  interconnects. 
In  addition, the  I/O  time  required  to  shift  the  subimages  into  and  out  of  the  subarrays 


limits  the  processor  performance.  Extracting  and  combining  information  from  multiple 
images  are  complex  tasks  which  further  limit  the  system  throughput. 

3)  Raster  Subarray:  A  raster  subarray  consists  of  a  special  memory  unit  to  produce  a 
complete  set  of  neighborhood  pixels  for  the  computation  unit.  The  input  image  is  loaded 
serially  in  a  raster  format.  Each  time  a  pixel  passes  through  the  first  register  of  the 
memory  unit,  the  previously  entered  pixels  are  shifted  along.  The  number  of  shift 
registers  per  line,  M,  is  equal  to  the  number  of  pixels  per  line  in  the  image.  A  minimum 
of  nine  registers  have  to  be  available  to  the  computation  unit  to  form  a  3  x  3  structure 
element  which  requires  a  total  of  3M  shift  registers.  The  output  is  in  same  raster  format 
as  the  input  with  in  delay  in  time. 

To  maintain  real  time  operation,  the  processor  can  be  pipelined  with  a  cascade 
of  parallel  subarrays  such  that  the  output  of  a  raster  subarray  becomes  the  input  to  a 
next  array  in  the  cascade.  The  number  of  subarrays  required  is  equal  to  the  number  of 
transformations  needed  to  perform  the  algorithm.  If  the  input  data  rate  is  slow  or  the 
processing  speed  is  sufficiently  fast,  the  number  of  cascaded  subarrays  required  can  be 
reduced  by  cycling  the  data  back  to  the  first  array  to  continue  the  processing  before  the 
next  stream  of  image  data  is  loaded.  ERIM’s  Cytocomputer  is  the  most  prominent 
processor  of  this  type. 

The  ideal  processor  architecture  in  terms  of  performance  is  the  parallel  full  array. 
All  image  pixels  are  transformed  simultaneously,  providing  a  tremendously  high 
throughput.  Its  implementation,  unfortunately,  is  not  feasible  with  current 
microelectronics  fabrication  technology.  The  inherent  massive  parallelism  of  an  optical 
processor,  however,  may  make  it  possible  to  implement  a  parallel  full  array  in  a 
reasonably  compact  package,  resulting  in  orders  of  magnitude  improvement  in  system 
throughput. 
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The  heart  of  an  optical  morphologic  image  processor  is  the  computation  unit 
which  is  composed  simply  of  an  input  SLR,  an  imaging  lens,  a  programmable  pupil  mask 
and  an  output  SLR  as  shown  in  Figure  3.4-1  which  performs  the  dilation  operation.  This 
basic  module  can  be  cascaded  and  arranged  in  a  feed  back  architecture  as  illustrated  in 
Figure  3.4-2.  The  data  circle  back  after  passing  through  and  processed  by  the  N  stages. 
The  processing  throughput  of  such  a  processor  is  maximized  when  the  time  required  to 
alter  the  transmittance  of  the  SLMs  is  matched  to  the  processing  time  through  the  N 
stages.  If  for  example,  the  switching  time  of  the  SLR  is  1  nsec  and  N  =  50,  the  SLM 
and  the  SLR  logic  must  be  programmable  within  50  nsec  to  keep  up.  Otherwise,  the 
processing  speed  must  be  slowed  down  or  the  number  of  processing  stage  has  to  be 
increased. 

The  pupil  functions  can  be  complex  and  it  may  be  difficult  to  obtain  an  SLM  with 
independent  amplitude  and  phase  control  and  high  switching  speed.  One  possible  design 
that  can  be  used  to  circumvent  to  problem  is  to  spatially  multiplex  the  needed  complex 
aperture  functions  on  a  transparency  and  make  use  of  the  fact  that  the  output  intensity 
distribution  of  an  incoherent  imaging  system  is  independent  of  the  spatial  position  of  the 
aperture  mask.  Placed  over  the  multiplex  aperture  mask  is  a  binary  SLM  which  blocks 
out  all  but  the  selected  aperture  function  as  illustrated  in  Figure  3.4-3.  As  the  data  are 
cycled  through  the  n  stage  optical  morphological  processor,  the  desired  aperture  function 
is  selected  by  controlling  the  on-off  pattern  of  the  binary  SLM.  Candidate  SLM  devices 
include  the  magneto  optics  SLMs  which  are  capable  of  ^sec  switching  speed.. 

Let  us  assume  that  the  structure  element  is  composed  of  3  x  3  neighborhood 
pixels,  a  single  transformation  will  require  a  minimum  of  9  multiplies  and  1  summation 
for  a  total  of  10  arithmetic  operations.  If  the  space-bandwidth  product  (SBWP)  of  the 
SLRs  is  256  x  256  pixels,  then  with  the  speed  assumed  in  Table  1-1  for  the  active  SLR 
devices,  the  processing  speed  of  the  optical  morphological  processor  will  be  25^  x  10 
ops/1  nsec  =  6  x  lO^**  ops/sec. 
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Figure  3.4-1.  Basic  Element  in  Optical  Morphological  Processor 
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A  rough  estimate  of  the  processor  size  is  about  5  cm  x  5  cm  x  10  cm  for  each 
optical  stage.  The  optics  of  a  10  stage  system  will  occupy  about  2500  cm^.  Adding 
another  10,000  cm^  for  the  control  and  driving  electronics,  the  total  processor  volume 
is  about  12,500  cm^.  The  processor  throughput  per  unit  volume  is  then  equal  to 
5.0  X  10^°  operations/sec/cm^. 

If  we  use  a  more  modest  and  realistic  processor  size  with  N=10  and  a  SLM 
switching  speed  of  10  fisec,  the  processing  speed  is  lowered  to  256^  x  10  x  10  ops/  10 
nsec  —  6.5  X  10^^  ops/sec.  The  processing  speed  per  unit  volume  achieved  with  these 
rather  conservative  parameters  is  then  equal  to  5.2  x  10^  ops/sec/cm^  which  is  still  very 
high. 

With  electronic  implementations,  the  small  structure  element  (e.g.,  3x3)  is 
typically  used  to  minimize  hardware  requirement.  Larger  structure  element  is  obtained 
through  successive  dilation  with  smaller  structure  elements,  taking  advantage  of  the 
distributivity  of  dilation.  With  an  optical  processor,  no  such  constraint  is  needed  and  a 
large  structure  element  can  be  used  directly.  The  dilation  by  a  large  structure  element 
can  be  performed  in  a  single  step,  further  increasing  the  system  throughput.  For 
example,  if  the  structure  element  is  composed  of  9  x  9  pixels  instead  of  3  x  3,  the  system 
throughput  is  increased  further  by  almost  an  order  magnitude. 

The  computation  unit  is  only  one  component  in  an  image  processor.  Other 
components  required  include  input  and  ouq)ut  interfaces,  image  memories,  controller, 
system  interconnects  and  decision  maker.  A  possible  system  architecture  of  a  special 
purpose  optical  morphologic  image  processor  may  operate  as  follows.  For  simplicity 
only  a  single  stage  is  shown.  The  input  image,  typically  from  video  source,  is  read  into 
two  frame  buffers  with  high  read  out  rate.  After  a  full  frame  fills  one  of  the  buffers,  it 
is  loaded  rapidly  into  die  SLR,  either  optically  or  electronically.  While  the  data  are 
being  loaded  into  the  SLR  and  processed  by  the  optical  processor,  input  image  data  are 


111 


directed  to  fill  the  second  frames  buffer.  This  is  done  to  ensure  there  will  be  no  loss  of 
data.  If  the  input  image  can  be  loaded  into  the  SLR  in  parallel  (e.g.,  a  direct  image  of 
the  scene),  then  a  simple  shutter  can  replace  the  dual  buffer  to  freeze  a  frame  of  data  to 
be  processed. 

The  SLRs  are  controlled  to  produce  either  the  direct  image  or  its  complement  at 
the  output,  and  the  threshold  level  is  set  by  the  control  computer.  The  SLMs  are 
programmed  by  the  controller  to  give  the  desired  pupil  function.  The  output  of  the  Nth 
stage  is  fed  back  the  first  processor  stage  to  continue  the  processing. 

The  optical  processor  filters  and  enhances  the  image  and  extracts  target  features. 
An  electronic  processor  has  to  examine  the  features  to  recognize  or  classify  the  targets. 
The  processed  optical  output  is  read  out  with  a  CCD  detector  array  and  digitizul.  The 
decision  making  will  be  perfOi  med  by  an  electronic  processor. 


3.4.1  ATC  Applications  for  Morphological  Processor 

Morphological  processing  has  been  successfully  used  in  many  difficult  image 
processing  applications,  including  ATC  and  ATR  with  low  contrast  infrared  images  and 
speckled  radar  images.  It  has  also  been  employed  extensively  in  medical  image 
processing  to,  for  example,  recognize  specific  cells.  Various  image  processing  functions 
related  to  ATC  can  be  implemented  with  a  series  of  morphological  operations  and 
transformations.  Some  example  of  simple  image  processing  functions  are  given  below. 

(1)  Differencing  or  intersecticn  can  be  used  for  change  detection. 

(2)  Opening  and  closing  can  be  used  to  remove  salt  and  pepper  noise, 
smoothing  and  size  filtering. 
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(3)  Difference  between  dilated  and  eroded  images  can  be  used  to  detect  the 
edges  of  an  target  image. 

(4)  Dilation,  hit  and  miss  transform  can  be  used  for  template  matching. 

(5)  Thinning  and  thickening  can  be  used  to  extract  and  enhance  certain  target 
features. 

(6)  Skeletonization  can  be  used  to  extract  target  features  and  to  establish  the 
connectivity  of  features. 

A  complex  imaging  processing  of  algorithms  can  be  implemented  with  a  series 
of  elementary  operations  or  transformations  in  the  form  of  an  algorithm.  For  example, 
to  perform  feature  extraction  or  dimension  reduction,  the  algorithm  may  involve  for 
example,  removal  of  salt  and  pepper  noise  to  smooth  out  the  image,  perform  size 
filtering  to  find  the  features  that  look  like  wheels  on  a  vehicle  and  to  locate  objects  that 
match  the  overall  size  of  the  target,  perform  a  skeletonization  to  extract  the  gross  features 
such  as  the  number  of  comers  in  the  target  image  or  to  determine  the  connectivity  of  the 
wheels.  To  implement  an  algorithm  may  require  the  performance  of  hundreds  of 
elenientary  operations  and  transformations  on  each  pixels.  To  be  able  to  perform  the 
algorithm  in  real-time  (video  frame  rate,  or  up  10^  pixels/ sec),  special  purpose 
processing  hardware  is  required. 

3.4.2  Summary  Comment  on  the  Optical  Morphological  Processor 

The  single  instruction  multiple  data  (SIMD)  processing  architecture  is  the  most 
efficient  architecture  for  morphological  image  processing  but  it  is  also  the  most  difficult 
to  implement  with  conventional  electronics  technology.  The  SIMD  architecture, 
however,  is  ideal  for  optical  implementation.  An  optical  processor  can  process  data  at 
a  very  high  rate  with  its  massive  parallelism  but  it  is  also  relatively  slow  in  changing  the 
instructions  to  individual  elements  in  the  processor  (e.g.,  reprogramming  an  SLM).  It 
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allows  a  large  amount  of  data  to  be  processed  simultaneously  with  the  same  instruction 
which  are  broadcasted  optically  to  all  processing  elements. 

We  have  shown  that  most  morphological  operations  and  transformations  can  be 
implemented  as  series  of  two  fundamental  local  operations  (complement  and  union)  and 
one  neighborhood  transformation  (dilation).  The  optical  implementation  of  these  three 
fundamental  operations  and  transformation  were  described.  An  optical  processing 
architecture  for  a  target  features  extractor  was  presented  around  which  an  automatic 
target  classifier  can  be  built. 

3.5  OTF  Synthesis  Preprocessor  and  Interferometric  Processor 

OTF  synthesis  preprocessing  and  interferometric  processing  are  well  established 
incoherent  optical  processing  techniques.  The  basic  operation  of  the  two  optical 
processors  will  not  be  changed  by  the  use  of  SLRs.  The  most  distinctive  feature  of  an 
SLR  based  system  is  ability  to  perform  bias  subtraction  on  tne  detection  device  which 
contrasts  with  conventional  implementation  using  a  detector  arrays  where  bias 
substraction  is  performed  external  to  the  detector.  Since  bias  build  up  is  the  primary 
problem  for  both  incoherent  optical  processors,  the  section  will  concentrate  on  addressing 
the  performance  of  an  SLR  in  bias  subtraction. 

Bias  build  up  limits  the  performance  of  an  incoherent  optical  processor  because 
it  occupies  the  bulk  of  the  dynamic  range  of  the  detector.  The  optical  output  of  the 
incoherent  optical  processor  can  be  modelled  as  B  ±  mB  where  B  is  the  bias,  m  is  the 
modulation  depth  of  the  signal  and  m<l.  Utilizing  the  full  dynamic  range  of  the 
detector,  B(l+m)  =  N  where  N  is  the  number  of  electrons  in  the  well  of  a  CCD 
detector  array,  we  have: 
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for  m  1  .  (48) 


SNR  =  =  2mi/Nr 

VEfl+m) 

The  SNR  can,  therefore,  be  improved  by  using  a  detector  with  a  larger  storage  capacity, 
N,  or  by  increasing  the  contrast  (m)  of  the  signal  or  both.  In  the  following  section,  the 
use  of  an  SLR  to  enhance  the  output  SNR  of  an  interferometric  processor  is  described. 

3.5.1  Spatial  Light  Rebroadcaster  for  Bias  Subtraction 

A  passive  SLR  typically  consists  of  a  uniform  layer  of  electro-optic  material  such 
as  an  electron  trapping  phosphor  [Lindmayarj.  Incident  writing  energy  at  short 
wavelength  (e.g.,  Xj  =  green)  is  absorbed  by  the  material,  exciting  electrons  to  the 
conduction  band.  The  electrons  then  fall  into  traps  where  they  are  stored.  When  the 
material  is  exposed  by  a  read  out  beam  at  a  longer  wavelength  (e.g.,  X,=near  infrared), 
the  trapped  electrons  are  excited  out  of  the  trapping  level  and  fall  back  to  the  valance 
band.as  shown  in  Figure  3.5-1,  emitting  light  at  wavelength  \  where  Xj  <  Xj,  <  X^. 

The  rate  at  which  electrons  are  filling  the  traps  is  determined  by  the  product  of 
the  write  beam  intensity  and  the  number  of  unoccupied  trap  sites.  If  the  input  radiation 
is  far  from  saturating  the  SLR  and  the  readout  depletes  only  a  small  percentage  of  the 
trapped  electrons,  the  intensity  of  the  output  emission  is  approximately  proportional  to 
the  number  of  occupied  traps  times  the  intensity  of  the  readout  beam.  The  products  of 
two  arrays  of  values  can  be  obtained  by,  for  example,  inputting  a  light  pattern 
representing  the  values  of  one  of  the  array  and  reading  it  out  with  a  light  pattern 
corresponding  to  the  second  array.  The  output  intensity  pattern  of  the  emitted  radiation 
is  proportional  to  the  products  of  the  two  arrays.  The  device  has  been  used  to  implement 
vector-matrix  multipliers  and  neural  networks  [Jutamulia,  McAulay].  What  we  are 
interested  in,  however,  is  the  dynamic  behavior  of  a  passive  SLR. 


115 


Figure  3.5-1.  Operation  of  Electron  Trapping  Material 


An  impetus  in  utilizing  an  SLR  for  bias  substraction  in  incoherent  optical 
processing  is  the  large  dynamic  range  provided  by  an  SLR.  The  number  of  traps 
available  in  an  SLR  is  much  larger  than  the  well  capacity  of  CCD  detector  arrays.  A 
thin  film  SLR  with  a  thickness  of  a  few  micrometers  can  have  as  many  as  10^  traps/^m^ 
[Lindmayar].  With  a  cell  or  pixel  size  of  lO/im  x  lOAim,  there  are  10®  traps  per  pixel. 
The  well  size  of  a  typical  CCD  detector  array  by  comparison  has  only  about  10® 
electrons.  Another  reason  for  using  an  SLR  is  its  simultaneous  write  and  erase  capability 
which  allows  the  bias  to  be  reduced  as  the  output  of  an  incoherent  optical  processor  is 
being  written  onto  the  device. 

3.5.2  Dynamic  Behavior  of  Passive  SLR 

The  rate  at  which  the  trap  sites  are  occupied  when  an  SLR  is  illuminated  with 
the  green  (Xj)  write  beam  is  given  by: 

dT_(x,t) 

— -  T„(x,t)]  («) 

where  is  the  absorption  cross  section  for  the  write  wavelength,  77 j  is  the  quantum 
efficiency  of  the  SLR  in  filling  a  trap,  W(x)  in  the  photon  flux  density  of  the  write  beam 
and  Tj  is  the  available  trap  density  (i.e. ,  total  number  of  available  traps  per  unit  area) 
and  T^(x,t)  is  the  density  of  occupied  traps  at  time  t.  The  solution  to  the  differential 
equation  is  given  by: 

TJx,t)  =  TJI  -  exp[-5i7,iW(x)  t  ]}  (50) 

assuming  the  initial  condition  of  T^(x,0)  =  0. 

In  the  case  of  read  out  by  an  infrared  (X^)  beam,  some  of  the  trapped  electrons 
are  excited  out  of  the  traps  and  fall  back  to  the  valance  band.  The  rate  that  the  trapped 
electrons  are  released  is  given  by: 


117 


dT,(x,t) 

=  8^rjJ^(x,t)R(x) 
at 


(51) 


where  5^  is  the  absorption  cross  section  for  the  readout  wavelength,  is  the  transfer 
efficiency  in  releasing  the  electrons  and  R(x)  is  the  readout  photon  flux  density.  The 
solution  to  the  differential  equation  is: 

T/x,t)  =  T,(x,0)  exp[-5oi,„R(x)  t  ]  .  (52) 

where  Tj(x,t)  is  the  number  of  traps  that  remains  occupieo  per  unit  area  and  T^Cx.O) 

0  is  the  initial  condition. 

If  we  first  write  on  the  SLR  with  a  beam  at  wavelength  \  over  an  exposure  time 
of  t'  with  a  flux  density  of  W(x)  and  then  read  out  the  SLR  with  an  IR  (\)  pattern  with 
flux  density  R(x)  for  a  time  period  of  t",  we  have  T,(x,0)  =  T^(x,t')  and  the  number  of 
traps  per  unit  area  that  remains  occupied  is  equal  to: 


T^(x,  t',  t')  =  T,{1  -  exp[-6iJjiW(x)t']}  exp[-a„77„R(x)(t')]  .  (53) 

The  final  readout  is  accomplished  with  an  uniform  IR  beam  with  photon  flux  density  I{(. 
The  emitted  photon  flux  density  is  given  by: 

lout(0  =  T(x,0)  exp[-5o»lolR  *  ]  (54) 

where  T(x,0)  =  T^(x  ,t',  t")  and  is  the  transfer  efficiency  of  the  device  in  converting 
energy  released  by  the  electrons  into  photons.  Detecting  the  output  of  the  SLR,  we 
have: 


J”  T(X,0)  Ve^oVoh  t  3 

”(x,0)77g{l  -  expC-a^Tj^lRT]} 


(55) 


where  r  is  the  integration  time  of  the  detector  array.  The  output  of  the  detector  array 
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is  linearly  proportional  to  the  distribution  of  the  trapped  electrons  remaining  in  the  SLR. 
In  the  following  section,  the  use  of  an  SLR  to  reduce  the  bias  in  the  output  of  an 
incoherent  optical  processor  is  described. 

3.5.3  Bias  Subtraction  in  Incoherent  Optical  Processing 

With  the  input  and  output  both  represented  by  light  intensities  which  take  on  only 
positive  real  values,  bias  build  up  is  a  significant  limiting  factor  for  incoherent  optical 
processing  systems  such  as  those  employing  OTF  synthesis  and  interferometric 
techniques.  Consider  an  interferometric  output  that  is  described  by; 

Io(x)  =  B  +  mB  cos(fx  +  B)  .  (56) 


If  we  add  a  second  channel  with  a  t  phase  shift  inserted  between  the  interfering 
wavefronts,  an  inverted  output  is  obtained.  That  is: 

I^(x)  =  B  -  mB  cos(fx  +  ^  .  (57) 

This  form  of  output  is  obtained  with  OTF  synthesis  utilizing  spatial  frequency  carrier  or 
two  pupil  synthesis  techniques.  Two  parallel  output  channels,  one  at  wavelength  Xj  and 
the  other  at  wavelength  Xj.  can  be  obtained  with  the  optical  arrangement  illustrated  in 
Figure  3.5-2  and  Figure  3.5-3  respectively. 

The  outputs  described  in  Eq.  (45)  and  (46)  also  represent  the  cosine  transform 
of  a  single  point  in  an  incoherent  input  field.  The  transform  output  obtained  with  an 
interferometric  processor  can  be  described  as  a  superposition  of  cosines  of  different 
spatial  frequencies  and  phases.  A  two-channel  interferometric  processor  that  provides  an 
inverted  set  of  outputs  at  wavelengths  and  \  is  shown  in  Figure  3.5-4. 
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IR/Green 

interlaced  Dichroic  filter 


Figure  3.5-2.  Two-Pupil  Synthesis  Interferometric  Processor  Using  an  SLR  for  Bias 
Subtraction 
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Figure  3.5-3.  A  Compact  Aperture  Synthesis  Interferometric  Processor 
Using  an  SLR  for  Bias  Subtraction 


Figure  3.5-4.  A  Two-Channel  Acousto-optic  Based  Interferometric  Processor 
Using  an  SLR  for  Bias  Subtraction 
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Let  W(x)  =  and  R(x)  =  Irl^(x).  From  Eq.  (11),  the  number  of  trapped 

electrons  after  t"  sec  of  destructive  readout  is  equal  to: 


T^(x,  t',  t')  =  T^(x,  t')  e 


-mBcos(f!c  *6)  1 


,j,  ^ -5ii}it'I,^[B+inBcos(fx+d)lj  ^ -6g»}pt'Ij[B-DQBcos(fx-t-5)] 


(59) 


where  and  1^  are  the  relative  brightness  of  the  write  and  read  beams.  Reading  out  the 
SLR  with  a  uniform  beam,  the  intensity  distribution  of  the  emitted  radiation  is  directly 
proportional  to  T(x,t',t")  which  is  plotted  in  Figure  3.5-5  for  different  values  of 
e  =  exp[-5oJ7oI,t'7^,i7jI^t']  and  B=l,  m=0.L  We  see  that  the  bias  is  gradually 
reduced  as  the  bias  removal  exposure  t"  increases.  The  trend  continues  until  e 
falls  below  0.3  where  the  signal  level  also  begins  to  be  reduced  by  the  bias  subtraction 
process.  In  Figure  3.5-6,  the  gain  in  signal-to-noise  ratio  is  plotted  against  the  erase 
exposure  as  expressed  by  e.  denotes  the  highest  SNR  that  can  be  achieved  without 
bias  subtraction.  Since  the  characteristic  of  an  SLR  is  nonlinear,  the  highest  SNR  is  not 
achieved  near  saturation  as  with  a  CCD  detector  array.  Instead,  the  optimum  exposure 
is  achieved  when  the  mean  or  bias  level  occupies  45%  of  the  available  traps.  With 
Tg^lO®,  Sm=982  for  an  input  signal  with  10%  contrast.  From  Figure  3.5-6,  we  see  that 
the  largest  improvement  in  SNR  is  obtained  when  c=0.29  where  the  gain  is  about  44%. 


Instead  of  performing  the  write  and  bias  subtraction  sequentially,  the  processing 
can  be  speeded  up  by  performing  the  input  writing  and  the  bias  subtraction 
simultaneously.  With  both  beam  patterns  illuminating  the  SLR  at  the  same  time,  the  rate 
of  change  in  the  number  of  trapped  electrons  is  given  by: 

IM  =  5i7,iW(x)[T,  -  T(x,t)]  -  5^i,„T(x,t)R(x)  (60) 

and  the  solution  to  the  differential  equation  is: 
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Figure  3.5-5 


Bias  Subtraction  as  a  Function  of  Erasure  Energy 
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(60) 


T(x,t) 


5i»7jW(x)T,  r  -[5,t,,W(x)  +  5o.)oR(x)J  t 

6i7?iW(x)-6„ijoR(x)^ 


assuming  an  initial  condition  of  T(x,0)=0.  Using  once  again  the  write  and  read  patterns 
given  in  Eq,  (14)  and  (15)  as  the  inputs,  the  outputs  of  the  SLR  are  plotted  in  Figure 
3.5-7  for  different  exposure  times  and  different  relative  beam  intensities.  The  SLR 
reaches  a  steady  state  after  a  certain  amount  of  exposure.  The  amount  of  SNR 
enhancement  achieved  when  the  SLR  reaches  steady  state  is  plotted  in  Figure  3.5-8  as 
a  function  of  the  erase-write  beam  ratio,  f  =  -5p7?oIjt"/5j77jI^t' .  The  largest  gain  in 
SNR  is  achieved  when  f  =  2  where  the  gain  is  about  57%,  The  SNR  is  zero  when 
there  is  no  erase  beam  because  the  SLR  is  driven  to  saturation. 


Due  to  vignetting  and  other  effects,  the  bias  in  the  output  of  an  incoherent  optical 
processor  output  is  typically  not  uniform.  Often,  the  amount  of  spatial  variation  in  the 
bias  term  is  larger  than  the  signal.  The  bias  subtraction  process  achieved  at  steady  state 
with  the  SLR  is  not  dependent  on  the  level  of  the  original  bias.  The  bias  subtraction 
method  produces  a  uniform  residual  bias  which  can  be  removed  by  simply  subtracting 
a  constant  from  the  output  of  the  detector  array.  This  is  illustrated  in  Figure  3.5-9. 
With  a  rectangular  input  window,  the  compressed  ouqjut  is  a  Sine  (sinrx/Tx)  function. 
In  Figure  3.5-6(a),  we  show  the  case  where  the  output  Sine  function  is  on  a  uniform 
bias.  The  bias  level  is  as  expected,  reduced  by  the  processing  with  the  SLR.  In  Figure 
3.5-9(b),  the  Sine  function  is  imbedded  in  a  low  frequency  nonuniform  bias.  The 
residual  bias  in  the  SLR  output  is  flat  much  like  the  case  in  Figure  3.5-9(a).  The 
uniform  bias  can  be  easily  removed  by  subtracting  a  constant  value  from  the  detector 
output.  In  Figure  3.5-9(c),  the  Sine  function  is  imbedded  in  a  high  frequency  non 
uniform  bias  which  makes  it  difficult  to  discern  the  presence  of  the  signal.  The 
processing  by  the  SLR  once  again  reduces  the  level  of  the  bias  and  makes  it  constant 
over  the  entire  output. 
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Sm=  Highest  SNR  achievable  with  bias  subtraction 
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Figure  3.5-9.  Removal  of  Space-Varying  Bias;  (a)  Uniform  Bias,  (b)  Nonuniform  Low 
Frequency  Bias,  (c)  Nonuniform  High  Frequency  Bias 
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With  the  proposed  processor  architecture,  the  SLR  reaches  a  steady  state  instead 
of  saturating  with  increased  exposure.  The  bias  subtraction  technique,  therefore,  adds 
robustness  to  the  incoherent  processing  system  as  well.  The  exposure  does  not  have  to 
be  controlled  precisely.  As  long  as  the  exposure  is  sufficiently  long,  an  enhanced  signal 
is  achieved. 

3.5.4  Projected  Performance  of  Acousto-Optics  Based  Interferometric  Processor 

Consider  an  acousto-optic  based  interferometric  processor  as  illustrated  in  Figure 
3.5-4.  Let  the  aperture  time  be  r,  N  be  the  number  of  pixels  (on  a  carrier  in  the  A-O 
cell  and  be  the  space-bandwidth  product  of  the  processor  output.  The  processing 
speed  of  the  processing  system  is  then  equal  to  M^N/t.  As  an  example,  with  a  Crystal 
Tech  40'’5  A-0  modulator,  the  carrier  frequency  fj.=75MHz,  the  bandwidth  BW  = 
50MHz,  T  =  80/xsec  and  M  =  N  =  4000.  The  processing  speed  is  then  equal  to  1.25 
X  10^^  op/sec.  The  optics  in  the  interferometric  processor  should  occupy  about  0.3  cubic 
feet.  Including  all  the  driving  electronics,  the  overall  processor  size  is  estimated  to  be 
approximately  1  cubic  foot.  The  predicted  system  performance  of  the  optical 
interferometric  processor  per  unit  volume  is  about  1.25  x  10^^  ops/sec/ft^  =  4.2  x 
10*  ops/sec/cm^. 

3.5.5  Assessment 

The  optical  interferometric  performs  the  function  of  an  array  processor  at  very 
high  speed.  The  large  bias  at  the  output,  however,  limits  the  performance  of  incoherent 
optical  processors.  A  processing  architecture  utilizing  a  spatial  light  rebroadcaster  as  an 
intermediate  detector  can  potentially  enhance  the  output  signal-to  noise  ratio  by  providing 
a  larger  dynamic  range,  removing  a  substantial  part  of  the  bias  and  all  of  its  non¬ 
uniformity.  By  using  the  dynamic  properties  of  an  SLR,  the  bias  reduction  technique 
also  provides  the  added  benefit  of  making  the  system  more  robust.  The  exposure  does 
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not  have  to  be  carefully  controlled  to  achieve  the  optimum  performance.  The  SLR 
reaches  a  steady  state  which  prevents  the  device  from  saturating  and  losing  the  signal. 
The  steady  state  can  be  reach  rapidly  if  adequate  laser  powers  are  available  for  writing 
on  and  reading  out  the  SLR. 

Incoherent  optical  processors  have  the  potential  of  utilizing  the  incoherent  image 
of  an  object  scene  directly  as  the  input,  bypassing  the  bottle  neck  created  by  spatial  light 
modulator.  However,  to  utilize  an  SLR,  the  image  of  the  input  scene  generally  cannot 
be  used  directly  because  the  natural  scene  may  not  possess  enough  of  the  desired  ratio 
of  energies  at  the  write  and  read  wavelengths  of  the  SLR. 
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4.0  TASK  3:  PRELIMINARY  EXPERIMENTS 


Preliminary  experiments  were  performed  to  evaluate  the  hardware  required  to 
implement  some  of  the  concepts  that  were  developed,  specifically  the  waveguides  of  the 
integrated  optics  artificial  neural  network  and  the  electron  trapping  material  manufactured 
by  Quantex  that  has  been  used  as  passive  SLR. 

4.1  Integrated  Optics  Architecture 

In  Section  3.2.3,  the  integrated  optics  architecture  for  artificial  neural  network 
processing  applications  was  developed,  analyzed,  and  a  point  design  given.  A 
proof-of-concept  device  was  then  designed,  partially  assembled,  and  preliminary 
experiments  performed.  It  was  found  that  many  of  the  components  needed  for  the 
proof-of-concept  device  were  available  off-the-shelf.  A  research  quality  waveguide  array 
with  the  couplers  did  need  to  be  fabricated,  however.  Preliminary  experiments  showed 
successful  coupling  of  light  into  the  waveguide  array,  but  were  only  partially  successful 
in  coupling  light  of  the  waveguides. 

4.1.1  Proof-of-Concept  Device  Design 

The  artificial  neural  network  application  chosen  for  demonstration  was  that  of 
determining  terrain  type  from  airborne  multispectral  imagery  of  the  ground.  Other 
ERIM  work  had  successfully  demonstrated  a  Kohonen  self-organizing  network  for  this 
application  [Kohonen].  The  specific  network  chosen  was  a  five  input,  five  node,  five 
output  Kohonen  network  operating  on  five  wavelength  bands  of  visible  to  short  wave 
infrared  data.  As  discussed  in  Section  3.2.1,  5-bit  accuracy  in  the  input  data  and  the 
weights  is  required  for  this  application.  From  the  earlier  work,  the  desired  weight  values 
were  already  known. 
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The  basic  design  of  the  integrated  optics  processor  is  shown  in  Figure  4. 1-1.  The 
off-the-shelf  components  will  be  described  first.  For  the  proof-of-concept  device,  the 
laser  diode  array  was  replaced  by  an  LED  array.  The  array  chosen  was  the  ROHM 
JA303012CL-01,  an  LED  print  head.  It  has  3,584  pixels  at  84.5-micron  spacing.  The 
emitting  area  of  each  pixel  is  50  microns  by  65  microns.  The  intensity  output  is 
0.83  microwatts  at  a  wavelength  of  0.66  microns  and  is  focussed  by  a  GRIN  lens.  The 
LEDs  can  be  modulated  at  up  to  5  MHz. 

The  LED  array  and  interface  electronics  were  connected  to  an  IBM  PC  and 
software  was  developed  to  control  the  array.  Because  the  LEDs  only  give  a  binary 
output  (on  or  off),  the  software  converts  the  5-bit  input  data  to  pulse-width  modulated 
form.  The  resulting  maximum  data  rate  is  100  KHz. 

An  EG&G  TB  series  linear  photodiode  array  was  chosen.  This  array  has 
128  photodiodes,  50  microns  in  width  by  2.5  mm  in  length.  A  fiber  optic  faceplate 
consisting  of  6-micron  diameter,  N.A.  =  1  fibers,  couples  light  onto  the  diodes.  The 
saturation  exposure  for  the  diodes  is  0.05  microjoules/cm^  resulting  in  a  saturation  charge 
of  29  picoCoulombs  and  a  dynamic  range  of  175,000:1.  The  maximum  readout  rate  is 
2.5  MHz.  This  array  was  also  interfaced  to  the  computer  and  controlling  software 
written. 

4.1.2  Waveguide  Array  Fabrication  and  Preliminary  Experiments 

To  match  the  characteristics  of  the  off-the-shelf  components,  the  waveguide  array 
was  designed  to  have  waveguides  50  microns  wide  by  65  microns  deep  by  2.5  mm  long. 
The  center  to  center  spacing  of  the  waveguides  is  84.5  microns.  The  waveguide  core  is 
made  of  Norland  optical  cement  (index  of  refraction,  n  =  1.56)  with  a  silicon  oxide 
bottom  and  side  cladding  layer  (n  =  1 .46)  and  a  methylsiloxane  polymer  top  cladding 
layer  (n  =  1.38).  The  initial  coupling  mechanism  chosen  was  to  place  diffuse  regions 
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Figure  4.1-1.  Integrated  Optics  Artificial  Neural  Network  Processor 
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in  the  top  surface  of  the  waveguide  to  scatter  light  toward  the  photodiode  array.  The 
mask  weights  would  be  implemented  by  area  encoding  the  diffuse  regions  and  would  be 
done  as  part  of  the  waveguide  fabrication  process.  Thus  there  would  be  no  separate 
mask  to  align  with  the  waveguide  array  during  the  experiment.  The  maximum  size  of 
an  individual  weight  was  chosen  to  be  10  microns  by  40  microns  which  easily  allows  the 
necessary  5-bit  array-encoding  accuracy. 

The  waveguide  fabrication  process  uses  microfabrication  technology.  Masks  were 
designed  and  made  for  the  two  photolithography  steps.  Mask  1  consists  of  long  parallel 
stripes  to  form  the  waveguide  pattern.  Mask  2  consists  of  the  known  weight  values 
area-encoded  in  five  by  five  arrays  as  well  as  some  test  five  by  five  arrays  with  non-zero 
weights  only  on  the  diagonal  of  the  array.  The  fabrication  process  consists  of  the 
following  steps: 

1.  Use  high  resistivity  (P  type  Boron,  1  to  3  ohm-cm)  intrinsic  <  110>  silicon 
substrates  polished  on  both  sides. 

2.  Grow  a  1.5-micron-thick  layer  of  thermal  oxide  on  the  substrate  in  a  furnace 
at  1000°  C  with  oxygen  and  water  vapor  for  20  hours. 

3.  Use  photoresist  and  Mask  1  to  place  the  waveguide  pattern  on  the  thermal 
oxide  layer, 

4.  Wet  etch  the  waveguide  pattern  through  the  thermal  oxide  using  NHsub4:HF 
for  28  minutes  and  remove  the  photoresist  with  stripper. 

5.  Wet  etch  the  waveguides  65  microns  into  the  silicon  using  a  hot  anisotropic 
KOH:Hsub20  etch  for  108  minutes. 

6.  Grow  a  1.5-micron-thick  layer  of  thermal  oxide  on  the  waveguide  wall  and 
bottom  surfaces.  This  is  the  cladding  layer  for  the  waveguide. 

7.  Fill  the  waveguides  with  Norland  61  optical  cement,  expose  to  UV,  and  cure 
the  cement  at  90°  C  for  12  hours. 

8.  Spin  on  a  0.5  micron  methylsiloxane  polymer  (Accuglass  512)  top  cladding 
layer  and  heat  cure  at  150°  C  for  24  hours. 
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9.  Use  a  1.5-micron-thick  photoresist  layer  and  Mask  2  to  place  the  weight 
mask  (area-encoded)  pattern  on  the  top  cladding  layer. 

10.  Reactive  ion  etch  through  the  cladding  layer  to  the  top  surface  of  the  Norland 
cement  waveguides.  This  process  leaves  a  diffuse  surface  on  the  Norland 
cement  in  the  weight  mask  regions. 

The  fabrication  process  described  above  is  the  product  of  a  collaborative  effort  between 
ERIM  and  WL/ELOT.  The  actual  fabrication  was  done  at  WL. 

Waveguide  arrays  were  made  at  WL  on  several  silicon  wafers  and  delivered  to 
ERIM  for  the  preliminary  experiments.  The  cross  sections  of  the  waveguides  were 
examined  under  a  microscope  and  found  to  be  of  the  required  size.  A  sample  photograph 
is  shown  in  Figure  4.1-2,  Light  was  coupled  into  the  waveguides  from  the  LED  array. 
Examination  of  light  leaving  the  far  end  of  the  waveguides  and  microscopic  examination 
of  the  top  surface  of  the  waveguides  showed  that,  for  the  most  part,  light  was  indeed 
coupled  into  and  contained  within  the  waveguides  and  propagated  to  the  far  end  with 
sufficiently  low  losses,  crosstalk,  and  stray  scattered  light  for  a  preliminary  experiment 
to  be  successful.  A  few  waveguides  did  have  cracks  in  the  Norland  cement  perpendicular 
to  the  direction  of  light  propagation  which  scattered  nearly  all  the  light  out  of  the 
waveguide  at  that  point.  The  incidence  of  these  defects  was  low  enough  so  that,  over 
the  many  redundant  arrays  fabricated  on  a  2-inch  wafer,  at  least  one  would  be  found 
which  was  defect-free. 

The  weight  masks  were  also  subjected  to  careful  examination  under  a  microscope. 
A  sample  photograph  is  shown  in  Figure  4.1-3.  It  was  found  that  the  diffuse  regions  did 
not  scatter  very  much  light  in  a  direction  which  would  be  within  the  acceptance  angle  of 
the  fibers  in  the  faceplate  of  the  photodiode  array.  (The  microscope  had  a  greater 
acceptance  angle  than  the  fibers.)  However,  light  propagating  in  directions  very  nearly 
along  the  length  of  the  waveguide  did  leave  the  waveguide  in  the  diffuse  regions,  strike 
the  nearly  perpendicular  wall  of  the  top  cladding  layer,  and  scatter  into  the  microscope. 
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This  effect  can  be  seen  in  Figure  4.1-3  as  narrow  lines  of  light  at  these  perpendicular 
walls.  This  light  appeared  to  be  modulated  only  by  the  width  of  the  weight  masks  and, 
therefore,  would  not  be  of  the  correct  value. 

Step  10  of  the  fabrication  process  was,  therefore,  modified.  The  substrate  was 
placed  on  a  45  degree  angle  during  the  reactive  ion  etching  to  create  walls  angled  at 
45  degrees  in  the  top  cladding  layer.  An  index  matching  fluid  (n  =  1.56  to  match  the 
Norland  cement)  would  then  be  used  to  couple  light  out  of  the  waveguide  in  the  weight 
mask  regions  (no  longer  relying  on  diffuse  scattering).  This  light  would  strike  the  angled 
wall  at  an  angle  of  approximately  45  degrees  to  the  wall  normal  and  be  reflected  toward 
the  fiber  faceplate  at  near  normal  incidence  to  the  faceplate.  WL  is  currently  fabricating 
this  new  waveguide  array  design.  When  ready,  it  will  be  used  to  continue  these 
preliminary  experiments. 

4.2  Commercial  Hiosphor  Based  Passive  SLR 

An  SLR  based  on  electron  trapping  phosphor  was  acquired  from  Quantex  Corp. 
and  its  read-write  capability  and  temporal  response  was  tested.  The  experimental  setup 
is  shown  in  Figure  4.2-1.  The  blue  (488nm)  beam  from  an  Ar"*'  laser  was  used  to  write 
onto  the  SLR  and  the  near  infrared  (l.Ob/tm)  beam  from  a  Nd;YAG  laser  was  use  to  read 
out  the  device.  To  assure  that  the  output  detector  detects  only  emission  from  the  SLR, 
the  laser  light  from  the  Ar"^  laser  was  spectrally  filtered  to  remove  any  orange  discharge 
glow  from  the  laser  tube  and  IR  and  blue  blocking  filters  were  used  at  the  output  to 
reject  the  read  and  write  beams.  The  read  and  write  beams  from  the  Nd:YAG  and  Ar'*' 
lasers  were  modulated  by  shutters  which  sent  a  write  pulse,  a  time  gap  and  then  a  read 
pulse.  The  SLR  emission  was  detected  by  a  photomultiplier  (PM)  and  monitored  by  a 
digital  scope. 
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There  are  two  types  of  emission  by  the  SLR,  spontaneous  fluorescence  caused  by 
the  blue  write  beam  and  photoluminescence  due  to  the  IR  readout  beam.  Fluorescence 
occurs  because  some  of  the  excited  electrons  spontaneously  fall  back  to  the  valence  band 
instead  of  into  traps.  Photoluminescence  occurs  when  the  trapped  electrons  are  excited 
by  the  IR  beam  out  of  the  traps  and  give  up  the  stored  energy.  The  communication  band 
is  quite  broad  and  some  of  the  fluorescence  emission  are  at  the  orange  wave];-ngth  of 
photoluminescence.  The  write  beam  must,  therefore,  be  turned  off  when  the  SLR  is  read 
out. 


In  Figure  4.2-2,  the  output  of  the  PM  is  shown  as  a  function  of  time.  The  write 
beam  was  turned  on  for  about  20  seconds  followed  by  an  8-second  time  gap  and  then  the 
read  beam.  When  the  write  beam  was  turned  on,  the  fluorescence  level  steadily  built  up 
with  exposure  time  because  electrons  were  excited  up  to  the  communication  band  at  a 
higher  rate  than  they  are  falling  back  fpontaneously  to  the  valence  band  or  into  the  traps. 
The  increase  in  electrons  population  in  the  communication  band  produced  a  stronger 
fluorescence  emission  with  time.  When  the  IR  read  beam  was  turned  on,  the  SLR 
emitted  via  photoluminescence.  The  emission  fell  off  exponentially  with  time  as  the 
population  of  trapped  electrons  was  depleted. 

If  a  short  IR  pulse  is  used  to  read  out  the  SLR  and  only  a  small  percentage  of  the 
trapped  electrons  are  released,  the  stored  information  can  be  readout  repeatedly.  The 
output  beam  brightness,  however,  will  decrease  exponentially  wi^h  each  subsequent 
readout.  Repeated  read  out  is  illustrated  in  Figure  4.2-3.  The  SLR  was  written  on  by 
a  strong  blue  pulse  from  an  Ar"*"  laser  and  then  readout  by  two  sequential  pulses  from 
a  Nd;YAG  laser.  The  first  pulse  in  Figure  4.2-3  was  due  to  fluorescence  when  the  SLR 
was  written  by  an  Ar"^  laser  and  the  subsequent  two  output  pulses  are  photoluminescence 
induced  by  the  read  pulses.  Some  of  the  trapped  electrons  were  released  by  the  first  read 
out  pulse  leaving  a  small  population  of  trapped  electrons.  As  expected,  the  intensity  of 
the  output  due  to  the  second  readout  pulse  was  slightly  lower  than  the  first. 
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Figure  4.2-2.  Photomultiplier  Output  in  Write-Read  Cycle 
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Figure  4.2-3.  Demonstration  of  Repeated  Readout 
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In  the  experiments,  several  drawbacks  of  the  electron -trapping  phosphor  material 
as  exemplified  by  Quantex’s  product  became  evident.  (I)  The  light  emitted  by  the  SLR 
is  very  dim.  One  reason  is  that  the  SLR  emits  uniformly  front  and  back  over  4t 
steradian.  A  f/2  lens  imaging  one-to-one  onto  the  detector  gathers  only  1/32  of  the 
emitted  light.  (2)  Rapid  erasure  of  the  stored  information  requires  a  large  amount  of 
energy  (approximately  200  MJ/cm^).  In  addition,  the  erasure  did  not  appear  to  be 
complete,  at  least  with  the  samples  we  obtained  from  Quantex.  This  resulted  in  a 
gradual  build  up  of  background  bias  with  repeated  write-read-erase  cycles.  (3)  The 
readout  is  destructive.  To  use  the  device  as  a  reference  mask  or  matrix,  for  example, 
the  device  must  be  written  periodically  to  refresh  the  memory.  There  is  also  a  trade  off 
between  output  brightness  and  the  number  of  times  the  stored  information  can  be  read 
out.  (5)  The  shift  in  wavelength  between  the  input  and  the  output  makes  it  impossible  to 
cascade  two  passive  SLRs  unless  they  can  be  doped  very  differently  such  that  the 
wavelength  of  emission  of  the  first  SLR  matches  the  wavelength  of  the  write  beam  of  the 
second. 
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5.0  CONCLUSION  AND  FUTURE  DEVELOPMENT 


The  Spatial  Light  Rebroadcaster,  particularly  of  the  active  type,  can  potentially 
be  a  powerful  device  that  can  serve  as  the  heart  of  a  compact  high  speed  processor.  The 
devices,  however,  are  still  in  a  very  early  developmental  stage  and  they  require 
significant  amount  of  further  development  before  they  can  be  used  competitively  in 
optical  processing  architectures  such  as  those  described  in  this  report. 

5.1  SLR  Performance  Requirements 

Passive  SLRs  such  as  those  implemented  with  electron-trapping  materials,  exist 
today.  Some  of  these  materials  were  developed  for  wavelength  down -conversion  to 
visualize  near  infrared  radiation  and  they  are  commercially  available.  The  performance 
of  these  passive  SLR  materials  and  devices,  however,  require  substantial  improvement 
in  several  areas  to  make  them  competitive. 

1)  The  slow  temporal  response  of  the  passive  SLR,  particularly  in  erasure,  limits  the 
cycling  rate.  The  throughput  achievable  is  too  slow  to  be  competitive  at  this  time. 

2)  Compounding  the  problem  of  low  cycling  rate  is  the  low  optical  efficiency.  The 
output  is  so  dim  that  the  output  must  be  integrated  over  a  significant  amount  of  time 
to  gather  enough  photons  to  provide  the  needed  signal  dynamic  range. 

3)  The  erasure  is  often  incomplete  unless  very  strong  light  or  heat  is  used.  The  need 
for  a  powerful  source  for  rapid  and  complete  erasure  impacts  negatively  on  power 
consumption. 

One  solution  to  the  problem  may  be  to  develop  an  SLR  that  emits  light  directionally 
(current  devices  radiate  isotopically,  over  4t  radian).  Improving  the  optical  efficiency 
would  allow  the  use  of  a  thin  layer  of  rebroadcasting  material  and  improve  the  cycling 
speed  of  the  device. 


In  addition,  there  are  inherent  characteristics  of  passive  SLRs  that  limit  their 
usefulness. 


1)  The  readout  is  destructive.  The  material  requires  constant  refreshing  to  keep  the 
data  stored  in  the  device.  A  trade  off  between  ouQjut  brightness  and  the  number 
of  number  of  times  the  stored  in  formation  can  be  readout  is  required. 

2)  The  input  and  readout  wavelengths  are  different  which  precludes  the  cascading  of 
devices  to  perform  sequential  operations  even  if  adequate  optical  efficiency  can  be 
achieved. 

3)  The  nonlinearities  exhibited  by  passive  SLRs  are  weak  and  they  cannot  be  easily 
changed.  The  type  of  operation  that  can  be  performed  is,  therefore,  restricted. 


Active  SLR  devices  have  the  inherent  flexibility  and  power  to  be  a  significant 
player  in  the  future  development  of  compact  high  speed  processing  systems.  They  may 
be  utilized  as  interconnects  and  as  the  processing  elements  in  an  hybrid. electronic/optical 
processor.  The  programmable  gain  and  nonlinearity  provided  by  the  device  are 
particularly  crucial  to  many  optical  computing  architectures.  The  development  of  these 
devices,  however,  are  still  in  an  early  stage.  Specific  area  that  requires  further 
development  includes  the  following. 


1)  Space-Bandwidth  Product.  The  advantage  offered  by  an  optical  processor  is  the 
massive  parallelism  of  the  computation.  This  advantage  can  be  resized  only  if  the 
space-bandwidth  product  of  the  input  and  output  devices  are  sufficiently  large. 
Devices  being  fabricated  at  this  time  are  very  small.  The  manufacturing  technology 
to  fabricate  a  large  array  with  acceptable  cost  and  yield  remains  to  be  developed. 

2)  Packing  Density.  The  most  attractive  promise  of  optical  processing  is  high  speed 
processing  in  a  small  physical  package  with  low  power  consumption.  To  fulfill  this 
promise,  the  large  space-bandwidth  product  must  be  accomplished  in  a  small 
package  that  draws  little  power.  Therefore,  the  device  size  must  be  small  and  the 
packing  density  must  be  very  high.  Considering  that  each  element  in  an  active  SLR 
is  consists  of  a  detector,  a  signal  conditioner  and  an  emitter,  a  3-dimensional 
structure  is  likely  to  be  required  to  achieve  the  density  desired. 
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3)  Addressing  Schemes.  To  maintain  a  high  throughput,  particularly  with  a  pipelined, 
recirculating  processing  architecture,  an  efficient  means  must  be  available  to 
address  and  program  the  elements  in  the  SLR  in  parallel. 

5.2  SLR-Based  Optical  Processors 

We  have  analyzed  several  optical  processing  architectures  that  utilize  either  active 
or  passive  SLRs  as  processing  elements,  storage,  input  and/or  output  devices.  The 
capabilities  of  those  that  employ  passive  SLRs  are  more  restrictive.  The  advantage  of 
using  a  passive  SLR  in  lieu  of  conventional  input,  output  and  storage  devices  such  as 
detector  arrays  and  electronic  memory  is  not  compelling  at  this  time.  The  lack  of  gain 
and  the  inability  to  be  cascaded,  in  particularly,  limit  its  usefulness.  The  potential 
strength  of  the  passive  SLR  lies  primarily  in  the  large  dynamic  range  it  can  provide.  It 
may  be  able  to  enhance  the  performance  of  systems  such  as  the  interferometric 
processors  where  this  feature  is  of  crucial  importance. 

The  active  SLR  is  essentially  an  integrated  array  of  elements  each  of  which  is 
composed  of  an  input  detector,  a  signal  conditioner  that  is  programmable  electronically 
or  by  the  output  of  a  second  detector,  and  an  emitter.  The  ability  to  provide  gain,  non¬ 
linearity  and  high  cycling  rate,  gives  the  active  SLR  unique  power.  It  can  perform 
simultaneously  the  functions  of  parallel  optical  interconnects  and  processing  elements. 
The  optical  quadratic  processor  described  in  this  report  is  an  excellent  example  of  the 
integration  of  these  processor  functions. 

So  far,  the  SLR  is  being  developed  as  a  general  purpose  device  that  can  be  utilized 
in  different  optical  processing  architectures.  The  system  performance  can  be  further 
optimized  if  the  SLR  is  designed  as  an  integral  part  of  the  processor.  With  such  an 
approach,  the  SLR  is  not  a  stand  alone  component  but  is  made  specifically  to  match  and 
operate  with  the  other  components.  The  integrated  optics  artificial  neural  network 
processor  is  a  good  example  of  this  design  approach. 
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It  is  recommended  that  future  development  of  SLR  based  optical  processors  be 
directed  towards  the  active  SLR  technology  and  optical  processing  architectures  that 
combine  the  interconnects  and  processing  elements  with  SLRs.  In  addition,  instead  of 
treating  the  SLR  as  a  generic  component  to  be  inserted  into  an  optical  processor,  the 
SLR  should  be  considered  as  an  integral  part  of  the  optical  processor  design.  The  pixel 
size,  number  and  spacing,  the  temporal  response,  the  packaging  and  integration  of  the 
SLR  should  be  custom  designed  to  match  all  other  components  to  maximize  the  processor 
speed  and  efficiency. 
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